Using real-world data to predict findings of an ongoing phase IV trial: glycemic control of semaglutide versus standard of care

We implemented this study using Optum’s deidentified Clinformatics Data Mart Database (July 2017 – May 2022) which includes commercial health plan data and Medicare Advantage members. This geographically diverse database spans all 50 states and provides comprehensive demographic information, enrollment status, inpatient and outpatient diagnoses, as well as procedure codes in the form of International Classification of Diseases, Current Procedural Terminology, and Healthcare Common Procedure Coding System codes. Information was also available on pharmacy dispensing records, including information on medication start and refill, strength, quantity, and days’ supply, for insured patients. Outpatient laboratory test results are available for a subset of beneficiaries whose labs are processed by a nationally operating laboratory test provider.

Emulating the SEPRA trial design,14 15 we established a cohort of adult T2DM patients who are on metformin monotherapy and who had a record of A1C either starting injectable semaglutide or a standard of care (SoC) antidiabetic medication. SoC included DPP-4is, SGLT-2is, or SUs, or GLP-1RAs other than injectable semaglutide, including exenatide.

The cohort entry date (CED) was defined as the dispensation date of injectable semaglutide (exposure) or SoC medications (comparator) between December 6, 2017 (following the approval of injectable semaglutide in the USA),19 20 and May 31, 2021. All eligible patients were required to have 180 days of continuous medical and pharmacy enrollment, during which there was no dispensation of either exposure or comparator medications, indicating new initiators.

The initial eligibility criteria for the SEPRA trial required patients to be on metformin monotherapy, excluding those using other antidiabetic medications. These eligibility criteria underwent multiple amendments, importantly allowing patients to be on one or two oral antidiabetic medications before being randomized. Given that the SEPRA trial protocol amendments were not public at the time that we preregistered our study protocol, the emulation’s eligibility criteria were designed to mimic the initial SEPRA trial eligibility criteria: being 18 years or older, having a diagnosis of T2DM and A1C value, being treated with metformin monotherapy, and started intensification of treatment to achieve glycemic control. Similar to the SEPRA trial, we excluded patients who were pregnant or had evidence of contraindications for semaglutide, including a record for advanced CKD, end-stage renal disease, dialysis, or renal transplant. More details are provided in online supplemental figure S1 and table S1, and operational details of the algorithms at https://osf.io/7smp9↗.

We assessed >50 patient characteristics in the baseline period of 180 days before the CED. The assessed covariates included demographic variables, calendar time, baseline A1C value, frailty score,21 combined comorbidity score,22 23 diabetes-specific complications, other comorbidities, use of medications, healthcare utilization indicators as a proxy for overall disease state, care intensity, and surveillance. A full list of covariates is provided in online supplemental table S2.

We emulated the trial’s primary outcome: the proportion of patients achieving A1C <7% at the end of year 1. We also assessed two secondary outcomes from the trial: the change in A1C (percentage points) from baseline to year one and the number of severe hypoglycemic episodes leading to inpatient admission or emergency room encounters, consistent with the trial definition of severe hypoglycemia. We assessed hypoglycemia based on diagnosis codes24 25 indicating hypoglycemia (Positive predicitive value (PPV) between 70–80%), either as the primary diagnosis in inpatient care settings or in any position in emergency department (ED) settings. The algorithm used is provided at https://osf.io/7smp9↗.

To mimic the SEPRA design, which allowed treatment intensification, switching, and discontinuation at the treating physician’s discretion, we conducted an as-started (AS) analysis as the observational analog to the intention-to-treat of the trial analysis as our primary causal contrast.15 For the A1C outcomes (ie, A1C <7% and change in A1C), the A1C value was evaluated in the 1-year outcome assessment window, that is, 275–455 days after CED, using the closest value to 365 days (online supplemental figure S2). The follow-up started from the beginning of the outcome assessment window and continued until the occurrence of the outcome, disenrollment from the health plan, death, nursing home admission, or the end of the study period. For the number of hypoglycemic episode outcomes, follow-up started 1 day after CED and continued until the occurrence of the outcome, disenrollment from the health plan, death, nursing home admission, or the end of the study period (May 31, 2022), whichever came first (online supplemental figure S3).

To mimic the high level of adherence usually seen in clinical trials, we secondarily conducted as-treated (AT) analyses as an exploratory secondary analysis. As the patients had an average follow-up time while on the treatment of 150 days in our AT cohort, the outcome assessment window was defined at around 5 months (91–212 days). The follow-up started from the beginning of the outcome assessment window and continued until the discontinuation of the drugs, or switch/augmentation of therapy, the occurrence of the outcome, disenrollment from the health plan, death, nursing home admission, or the end of the study period. We defined discontinuation as a gap of more than 60 days following the last day of supply of the exposure/comparator or metformin therapy. Switch/augmentation was defined as adding other antidiabetic agents beyond the one initiated on the CED (see). online supplemental figures S2 and S4

We used 1:1 nearest-neighbor propensity score (PS) matching without replacement and a caliper of 0.01 of the PS distribution to balance >50 patient characteristics across treatment groups. The PS was estimated by fitting a logistic regression model without further variable selection (online supplemental table S2). https://osf.io/gjmuv↗ provides the R code used to create the PS matching cohort. We assessed the postmatching balance via: (1) the c-statistic, expecting 0.5 for perfect balance in measured patient characteristics, and (2) the standardized difference between the characteristics, ≤0.1 indicating good balance.26

After matching, our feasibility counts substantially exceeded the trial’s intended recruitment of 1,387 patients, which was estimated to provide 90% power to confirm the superiority.14 Therefore, we inferred that our study would have at least equal power to the SEPRA trial. For the outcomes of A1C <7% and change in A1C, we imputed the missing A1C value for those patients who entered the 1-year outcome assessment window but did not have A1C measurement using multiple imputation by chained equations (MICE) in combination with a random forest approach to imputing missing data (online supplemental methods 1 for details).27 28

A detailed protocol of our study was registered on clinicaltrials.gov on October 10, 2022, before conducting inferential analyses. Because SEPRA’s registration did not specify how the proportions of patients with A1C <7% would be compared between the treatment arms, we chose a relative risk estimate.29 30 The change in A1C from baseline to follow-up was calculated as the difference between follow-up and baseline A1C values in percent points. All analyses were performed using Aetion Evidence Platform V.5.5 (Aetion), which has previously been validated by accurately repeating a range of previously published studies,31 32 STATA V.17 (StataCorp), and R V.4.4.1.

We conducted several prespecified subgroup analyses, assessing A1C outcomes using each of the four drug classes included in the SoC as separate comparator groups.

To evaluate the robustness of our missing data imputation results, we conducted multiple sensitivity analyses for both our AS and AT causal contrasts. We conducted a complete case analysis where we assessed the A1C outcome only among patients with A1C values in the outcome assessment window. In addition, we used different imputation methods, including MICE33 34 and fixed imputation of values at different points of the observed distribution of A1C in follow-up as a tipping point analysis, as well as a delta-adjusted missing not at random sensitivity analysis to test the impact of lower A1C values35 (online supplemental methods 2).

To assess whether trial emulation would lead to the same regulatory conclusions as the interim results of the ongoing RCT, we computed three predefined binary metrics1: statistical significance agreement, where estimates and CIs are on the same side of the null2; estimate agreement, where the trial emulation estimates fall within the 95% CI of the interim trial results and3 standardized difference agreement, which compares the treatment effect estimates between the trials and their emulations.13 36 We were able to calculate concordance for our primary outcome (A1C <7%) but could not do so for the change in A1C percentage points due to the lack of SD reporting in the interim results.

This cohort study used longitudinal claims data with lab test results to predict the results of the pragmatic SEPRA trial. Our analyses indicated that patients initiating injectable semaglutide achieved better glycemic control and a greater reduction in A1C compared with those on SoC medications. These findings are comparable with the now-public interim trial findings—with approximately 30% more patients in our study achieving A1C <7% at 1 year after initiating injectable semaglutide compared with SoC. However, both groups were effective at lowering A1C, and we observed that semaglutide reduced A1C by only 0.2% more than SoC, a relatively small improvement that may not translate to large effects on patient complications at a population level.

Due to lower-than-expected recruitment, multiple amendments were made to the SEPRA trial, reducing the original target enrollment of 2,250 by 43% to 1,278, resulting in wider-than-expected CIs. In contrast, our real-world study was able to emulate the original trial question with a sufficiently large sample size, offering a complementary perspective. This adds to the growing body of evidence that the role of well-designed studies using RWD in supporting clinical trials—particularly when trials face challenges such as limited recruitment, protocol amendments,15 or early termination.18 In such scenarios, RWE can help bridge evidence gaps and provide timely insights into effectiveness and safety in broader patient populations. It also enables the examination of questions related to effectiveness and safety in different populations. Despite our findings aligning with the interim results of the SEPRA trial, we recognize that the final results may differ, and we await the full release of the trial results to assess how closely our predictions match the final results.14

Although we were unable to exactly emulate the final exclusion criteria and analysis plan of the SEPRA trial in our pre-specified protocol (because these details were not public during our protocol development stage), a key strength of this study is the registration of a fully specified protocol, including study design and analytic choices, prior to public release of the trial results. This approach ensures that our design choices are not influenced by knowledge of the trial outcomes. We were also able to clearly describe post-hoc analyses to better align the statistical analyses used in our RWD study with the analyses for the SEPRA trial after the trial’s statistical analysis plan was made public. To our knowledge, this is the first study designed to use RWD to emulate and predict the results of the SEPRA trial. Our study used contemporary statistical methods and generated robust estimates across an array of prespecified sensitivity and subgroup analyses.

Despite employing rigorous analytic methods, residual confounding due to unmeasured factors cannot be entirely ruled out. A major strength of our work lies in the detailed handling of partially observed A1C measurements. Missing data diagnostics suggested that the missingness mechanism likely follows the missing at random assumption,27 making multiple imputation a preferred method over complete case analysis. This approach offers several advantages, as it retains all patients in the analysis and provides more realistic SE estimates by accounting for both conventional sampling variance (within-imputation variance) and the additional variance introduced by missing data (between-imputation variance).44 Additionally, we conducted multiple sensitivity analyses to demonstrate the robustness of our findings.

As noted earlier, only a subset of patients in our database had laboratory data available, and our inclusion criteria and outcomes were based on the availability of A1C data. However, the availability of this data is driven by lab provider partnerships for Optum Clinformatics. We use our inclusion criteria to balance the availability of labs at baseline, but in routine clinical practice, the frequency and timing of A1C assessments are influenced by various factors, including the type of antidiabetic treatment regimen, A1C level at treatment initiation, the anticipated change in A1C, and adherence to testing requirements.45 As a result, limiting our analysis to patients with lab data may impact the generalizability of our findings. In addition, some factors that impact glycemic control, such as lifestyle modifications, duration of diabetes, or body mass index, are not well captured in claims databases and, therefore, could not be accounted for in our analysis. Of these, the duration of diabetes is especially important, given its link to progressive β-cell decline and reduced effectiveness of oral antidiabetic medications over time.46 However, the potential bias due to lack of direct measurement of duration of diabetes is mitigated in our study design, which compared initiators of the first second-line therapy started after metformin monotherapy, so patients in both groups likely had a similar duration of diabetes. In the clinical trial, patients paid a maximum of US$20 per month in copay for their medication, with any additional costs reimbursed, regardless of whether they were in the semaglutide or SoC group.15 However, in our study, patients were responsible for the full copay, which may vary across insurance plans and thereby contribute to channeling of the patient population as well as differences in adherence. Although we adjusted for proxies of socioeconomic status (eg, race, copay amount, insurance type, low-income indicator that were available in the data47 48), there may still be residual confounding from unmeasured factors such as income or health literacy. The direction of this bias is uncertain. Clinicians might preferentially prescribe semaglutide to patients with higher socioeconomic status or who are perceived as more adherent (potentially overestimating its benefit). Alternatively, clinicians might choose to prescribe semaglutide over some alternative SoC therapies for frail older adults of lower socioeconomic status due to concerns regarding adverse effects like hypoglycemia (which would bias the treatment effect toward the null). Additionally, the composition of patients in our study may differ from participants in the SEPRA trial, which may contribute to divergence in results in the presence of effect measure modification. As noted in the results section, there were notable differences regarding baseline characteristics between our routine care population and SEPRA participants. Furthermore, we observed variability in effect sizes observed in subgroup analyses comparing semaglutide to specific classes of drugs contributing to the SoC arm. Additionally, the distribution of glucose-lowering therapies that the SoC arm in the SEPRA trial and the switching patterns over follow-up are not yet publicly available. Differences in such factors between our study and the SEPRA trial could affect the results. However, we note that we observed similar results in our AS analysis that did not censor on treatment switching and our AT analysis, which did censor on switching between arms as well as within class in the SoC arm. Our study includes only patients with commercial insurance in the USA, which may limit the generalizability of these findings to individuals without insurance, those covered by public insurance plans, or populations outside of the USA. Finally, severe hypoglycemia was identified using diagnosis codes from ED visits or inpatient encounters. Although this approach aligns with the SEPRA trial definition, the diagnostic codes used to define severe hypoglycemia have been reported to have a PPV of 70–80%.24 25 Any resulting outcome misclassification could be nondifferential, which would bias results toward the null, or differential, creating uncertainty in the direction of potential bias.

Data may be obtained from a third party and are not publicly available. Access to the de-identified source data may be granted to authorized researchers via application and licensing with the data providers: Optum Clinformatics (V8.1, data range from Q1_2004 to Q2_2022), connected@optum.com, https://www.optum.com/business/solutions/life-sciences/real-world-data/claims-data.html↗.