Data source
The Clinical Practice Research Datalink (CPRD) Gold database, which has over 19 million patient records in the UK from over 940 participating general practices, was used. The CPRD pregnancy register, which captures information from maternity, antenatal, and delivery records, was used to identify pregnancies within CPRD GOLD.
Study population
The target population was women who had been pregnant aged 15 to 49 years who were registered with their GPs between January 2000 and December 2021 with linkage to the Hospital Episodes Statistics (HES). To ensure sufficient quality data at baseline, participants contributed to the cohort after a minimum registration period with their practice of at least a year. Women were followed up from 15 months after date of conception (approximately 6 months postpartum) of the current pregnancy (i.e. for women with more than one pregnancy, the last pregnancy was used), regarded as the index date, to allow for normal physiological changes of pregnancy to resolve and allow time for postpartum information to be recorded in the primary care database [13, 14]. Women were followed until the earliest of outcome date (diagnosis of cardiovascular disease), transfer date from the practice, last date of practice data collection, date of death or study end date. In the absence of any of the above events, participants were censored 10 years after the index date. Women with pre-existing CVD or on statins before the index date were excluded.
Predictor variables
Traditional predictors
The traditional risk factors of CVD were obtained from the QRISK®3 algorithm [15]. These were age, ethnicity, deprivation (quintiles of Townsend score), systolic blood pressure (SBP), standard deviation of at least two SBP measurements, body mass index (BMI), total/HDL cholesterol ratio, smoking status, family history of CVD in a first degree relative aged less than 60, diabetes, rheumatoid arthritis, atrial fibrillation, chronic kidney disease, diagnosis of migraine, corticosteroid use, systemic lupus erythematosus, atypical antipsychotics, current treatment for hypertension (at least one of thiazide, β blocker, calcium channel blocker, or angiotensin converting enzyme inhibitor), and diagnosis of severe mental illness. Similar to QRISK®−3, medications (treatment for hypertension, corticosteroids and atypical antipsychotics) were measured as at least two prescriptions before the index date with the latest prescription recorded within 28 days of the index date. For all the other predictors, the latest information recorded in the general practice before the index date was obtained.
Additional pregnancy-related candidate predictors
Several pregnancy and reproductive-related factors were identified from an umbrella review on the associations of reproductive factors with CVD and from discussions with clinicians and patient research partners [12]. These included polycystic ovary syndrome, pre-eclampsia, small for gestational age, postnatal depression, endometriosis, irregular menses, gestational diabetes mellitus, gestational hypertension, miscarriage, preterm birth, placental abruption and number of previous pregnancies [16]. The pregnancy-related candidate predictors were measured as any history of the pregnancy complication from previous pregnancies (e.g. history of gestational diabetes mellitus before the current/last pregnancy).
All candidate predictors were evaluated to quantify missing data, identify outliers and ensure the correct measurement units were used. Definitions of the candidate predictors are provided in Additional file 1: Table 1 [17,18,19].
Outcome
The outcome of this study was the first recorded diagnosis of cardiovascular disease (coronary heart disease, stroke, myocardial infarction, or transient ischemic attack). This definition was based on the QRISK®−3 algorithm’s definition of CVD to ensure comparability of the updated models [20].
Statistical analysis
Missing data
For the external validation of QRISK®−3, the approach used to handle missing data at the implementation of the algorithm was adopted. Missing systolic blood pressure, body mass index, and total/HDL cholesterol ratio were imputed based on age and sex using single imputation in line with recommendations from the recent literature [21, 22]. Missing smoking status was assumed to be non-smoker, ethnicity was assumed to be white, and missing deprivation scores were imputed using the median value. Missing entry of a condition was taken to indicate absence of the condition (e.g. missing diabetes record was taken to mean no diabetes).
For the development of updated models, candidate predictors with more than 40% missing data were excluded; otherwise, the above single imputation approach was used. A table with proportion missing for each variable and method of handling the missing data is provided in Additional file 1: Table 2.
Evaluation of QRISK®−3 in external data
The first objective was to evaluate the QRISK®−3 algorithm in the population of women who had been pregnant to assess the performance of the risk equation in this cohort. This formed the benchmark for models with additional pregnancy-related predictors.
We calculated the 10-year predicted risk of CVD in the cohort using the QRISK®−3 women’s risk Eq. [15]. The 10-year observed risk was obtained using a pseudo-value approach [23]. The performance of the model was then evaluated using measures of discrimination (the model’s ability to differentiate between those who developed CVD and those who did not) and calibration (agreement between predicted and observed risk). Discrimination was quantified using Harrell’s C statistic, time-dependent C statistic and Royston’s D statistic. Calibration was quantified by plotting the 10-year observed probability of CVD against the 10-year predicted probability of CVD using the “pmcalplot” package in Stata using the default 10 equal risk groups based on percentiles [24]. In addition, summary measures of calibration (calibration-in-the-large, calibration slope and calibration intercept) were estimated. Mean calibration (calibration-in-the-large), which measures the agreement between predicted and observed survival probability, was estimated as the ratio of the observed survival probability (Kaplan–Meier estimate of experiencing CVD at 10 years) and the average predicted risk at 10 years [25]. The calibration intercept was calculated by fitting a generalized linear model of pseudo-values as the outcome and the predicted risk estimates (transformed with complementary log–log function) as an offset. The intercept from this model indicates the predicted risk is too high if the intercept is negative and too low if the intercept is positive [26, 27]. The calibration slope was estimated by fitting a similar model to that used for the calibration intercept but allowing the coefficient for the (complementary log–log) transformed predicted risks to be estimated. The coefficient of the transformed predicted risk estimates is the calibration slope [26, 27].
The clinical utility of the model was assessed using decision curves considering a range of risk thresholds up to 10% [28, 29]. We used the ‘dcurves’ package to visualize net benefit and plotting the decision curve. We used vector of threshold probabilities between 0 and 1 with the default sequence by 0.01 [28, 30].
Model update: re-calibrating the baseline risk of QRISK®−3
To assess whether the predictive performance of QRISK®−3 could be improved by re-estimating the baseline risk in the cohort of younger postpartum women, we re-calibrated QRISK®−3 using the 10-year baseline survival value estimated in the cohort by forcing the predictor effects to be the same (fitting the survival data to the QRISK®−3 linear predictor as an offset using Cox regression model) and re-assessed the performance of the re-calibrated model.
Model development and evaluation of updated models
After evaluating the performance of the QRISK®−3 algorithm (the benchmark model), three new models were developed and internally validated; Model 1 included the QRISK®−3 linear predictor (obtained from external validation step) plus pregnancy-related factors as predictors, Model 2a included QRISK®−3 predictors only (without interaction terms) to re-estimate QRISK®−3 coefficients and lastly Model 2b included QRISK®−3 predictors plus pregnancy related factors.
The primary timepoint of interest for the risk prediction models was 10 years in line with NICE guideline recommendations for interventions based on the 10-year risk of CVD [31].
Cox proportional hazards regression was used to develop the new models following practical approaches for risk prediction models [32, 33]. The accompanying 10-year baseline survival for each model was estimated non-parametrically using the Breslow method. The initial model included all the candidate predictors, and then variable selection was performed using the least absolute shrinkage and selection operator (LASSO) to determine predictors included in each model [34, 35]. The QRISK®−3 predictors were forced to remain in the model. After variables were selected, the final model was then fitted using Cox regression with the selected additional predictors. The continuous variables were included in the models on their continuous scale, with non-linear relationships with the outcome modelled using fractional polynomial terms. The fractional polynomial terms for the continuous variables were obtained based on complete data similar to QRISK development [15] and the resulting terms were then used in developing the updated models, including variable selection using LASSO. Internal validation was performed using 500 bootstrap samples to account for overfitting and estimate optimism, repeating the modelling process in each bootstrap sample and comparing performance in the bootstrap sample and original data to obtain optimism-adjusted statistics. Measures of discrimination and calibration were used to evaluate the new models and were compared with the performance of QRISK®−3. All analyses were conducted in R statistical software, R version 4.2.1 and in Stata.
Sample size
Determination of sample size for external validation and development of the new models was detailed in the protocol for this study [16]. Briefly, we established that a minimum sample size of about 24,000 women and 264 CVD events would result in precise estimates of model performance, for example with a calibration slope CI width of 0.3 (i.e. CI width of 0.85–1.15 assuming the true value is 1), with an assumed 20% censoring rate by 10 years [36, 37].
Sensitivity analysis
Because QRISK®−3 was developed for those aged 25 to 84 years, we carried out sensitivity analysis to compare the performance of QRISK®−3 with and without women aged below 25 years to assess the impact of applying the model outside the age group included in the development of the model. We also repeated the analysis in complete data (patients without missing data in the predictors). We also repeated the analysis after using multiple imputation with chained equations to impute variables with missing data. Multivariable imputation with chained equations was performed to generate 20 imputed datasets for missing BMI, SBP, total cholesterol: HDL cholesterol ratio (TC: HDL), systolic blood pressure (SBP), SBP standard deviation and smoking status. Performance measures were pooled across the imputed datasets using Rubin’s rules [38].
Model presentation
This study has been reported following the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD + AI) guidelines (Additional file 1: Table 3) [20, 39].