What this is
- This research evaluates the clinical benefits of immune checkpoint inhibitors approved by the FDA.
- It uses the ESMO-MCBS and ASCO VF frameworks to assess the value of these treatments.
- The study includes data from pivotal randomized controlled trials (RCTs) conducted between 2011 and 2018.
Essence
- All FDA-approved immune checkpoint inhibitors based on RCTs meet the ESMO-MCBS threshold for clinical benefit, with a median Net Health Benefit (NHB) of 55.3.
Key takeaways
- Eighteen pivotal RCTs were assessed, with all meeting the ESMO-MCBS meaningful benefit threshold. Eight trials received the highest grade of five, indicating strong clinical benefits.
- The median NHB for agents with an ESMO-MCBS grade of five or A was 56.8, suggesting significant overall survival advantages.
- Twelve trials demonstrated improved toxicity profiles, with only one trial reporting increased toxic deaths, indicating a favorable safety profile for most treatments.
Caveats
- Toxicity data were limited, as only adverse events occurring in at least 10% of patients were reported, potentially underestimating the true toxicity rates.
- Patient-reported outcomes (PROs) were not available for all agents, which may affect the perceived clinical benefit.
- The analysis focused solely on immune checkpoint inhibitors without direct comparisons to other approved therapies, limiting broader context.
Simplified
Background
Knowledge of the potential benefits and risks associated with the use of anticancer therapies is fundamental for making treatment-related recommendations and decisions. Two important oncology societies have recently taken a step forward to quantize the clinical benefit. The American Society of Clinical Oncology (ASCO) Value Framework (ASCO-VF) [1], which was updated in 2016 [2], and the European Society for Medical Oncology developed its Magnitude of Clinical Benefit Scale (ESMO-MCBS) for drugs indicated in the treatment of solid cancer [3], which also updated in 2017 [4]. They have been used to grade US Food and Drug Administration (FDA)-approved new drugs for treating advanced solid cancers [5–7]. In the study by Vivot and colleagues, they found that Many recently FDA-approved new cancer drugs did not have high clinical benefit as measured by ASCO-VF and ESMO-MCBS.
The growing wave of progress using cancer immunotherapy, which has extended and improved the lives of patients, many of whom had few other effective treatment options has yielded high expectations from all stakeholders. However, there are also concerns about the value of check point inhibitors. Many immune checkpoint inhibitors were approved based on single-arm studies, only recently more RCTs were finished and reported.
Patient-reported outcomes (PROs), such as symptoms, quality of life (QOL), and patient-perceived health status supplement clinical data and are now more important during decision-making in oncology because they provide a holistic understanding of patient experience and treatment effectiveness [8, 9]. Both ESMO-MCBS and ASCO VF incorporated QOL into the determination of the value of a treatment. ASCO VF awarded bonus points for treatment with a statistically significant improvement in cancer-related symptoms. However, PROs usually were not reported in the primary report or approval documents, but subsequently reported as separate articles.
In this study, we aimed to describe the clinical benefit of checkpoint inhibitors that were recently approved by the FDA based on RCTs using ESMO-MCBS and ASCO VF, and whether these agents reach defined thresholds of long-term benefit in the two value frameworks. We also compare the values based on primary reports with those assessed based on updated reports including long-term survival reports and/or QOL reports.
Methods
Data sources
We identify all approved indications of immune checkpoint inhibitors (Ipilimumab, Nivolumab, pembrolizumab, Atezolizumab, Avelumab, and Durvalumab, Cemiplimab) between January 1, 2011 and September 30, 2018 by searching FDA website [10]. Only indications approved based on RCTs were included and those approved based on single arm trials were excluded. Indications that were granted accelerated approval based on single arm trials but subsequently obtained regular approval with positive confirmatory RCTs were included. We included drugs used both in the metastatic setting and adjuvant setting of treatment of solid tumors.
Data extraction
Information including medians and HR of OS (PFS or DFS) and 95% CI, grade 3 or 4 toxicities in each arm, QOL data, survival probability at fixed or specified time were extracted from the reports of pivotal clinical trials supporting the FDA approval and FDA documents (drug labels and review summary retrieved from Drugs@fda website [10]). Survival probability at fixed or specified time was extracted directly from Kaplan-Meier curves using digital software (DigitizeIt). Baseline characteristics such as drug name, indication, trial name, sample size, primary outcome, tumor type, year of approval were also collected. When statistically significant results were reported for more than experimental arms, then each arm was evaluated separately and assigned a separate grade.
ESMO-MCBS and the ASCO-VF
ASCO-VF and ESMO-MCBS both quantify treatment benefit in a survival endpoint. ESMO-MCBS grade was assigned based on the lower limit of the 95% confidence interval of the hazard ratio (HR), and in conjunction with the minimum absolute gain differences in median survival or by the increase in survival at a fixed time, and further adjusted on QOL, toxicity and long term plateau of survival curve. ESMO-MCBS grades, in the non-curative setting, range from 1 to 5, with grades 4 and 5 representing meaningful clinical benefit, in the curative setting, range from A to C, with A and B representing meaningful clinical benefit. ASCO-VF score was assigned primarily on the point estimate of the HR with adjustment on toxicity and bonus points including tail of the curve, palliation, QOL and treatment-free interval. ASCO-VF score is continuous with a higher score representing a better score, and no cut-off value was provided to define clinical benefit. Both value frameworks incorporated amendments to introduce tail-of-the curve credits for progression-free survival and overall survival. For ESMO-MCBS, credit is given for a 10% or greater absolute gain at prognostically weighted specified time points in the true tail of the curve. Grading based on “long term” survival points differs depending on a PFS or OS endpoint (i.e., for PFS, this is an upgrade, while, for OS, this is an additional grading using the curative framework, e.g., 4/A). None of the trials actually meet this OS upgrade given the length of time required for the data to mature. ASCO-VF awarded 20 points of tail-of-the-curve bonus points if, at twice the median survival time (or DFS) in the control arm, there was an improvement of at least 50% in survival provided the survival in the control group was at least 20% and award 16 points (0.8 × 20) if the improvement is in PFS. ASCO-VF further awarded treatment with a statistically significant improvement in cancer-related symptoms (10 points) or improvement in treatment-free interval (10 points).
Two review authors (F.L. and S.Z.) independently scored each indication using ESMO-MCBS and the ASCO-VF with discrepancies resolved by a third investigator. We used the k coefficient to determine degree of agreement between reviewers. For trials with two or more immunotherapy arms, we scored each arm separately, but only the arm with higher score was used to represent the value of the specific indication in all analysis.
Updated value score
Value of approved drugs may change as long-term follow-up data or QOL data (which is usually not available or reported when initially approved) become available. Particularly, drugs that failed to qualify the tail of the curve bonus due to limited follow-up time can show long term plateauing of survival with longer follow-up time.
We searched latest drug label or PubMed to identify if updated reports of survival, toxicity or quality of life data and assigned updated score for these indications. When multiple reports of updated reports of survival were published, the most up-to-date one was used.
Results
| Approved drug | Indications | Pivotal Trial | Primary endpoint | Sample size | Year of approval |
|---|---|---|---|---|---|
| Pembrolizumab plus chemotherapy | First-line therapy of metastatic non-squamous NSCLC | KEYNOTE-189 | OS and PFS | 616 | 2018 |
| Nivolumab plus ipilimumab | First-line therapy of intermediate or poor risk advanced renal cell carcinoma | CHECKMATE-214 | OS, ORR and PFS | 847 | 2018 |
| Durvalumab | Consolidation therapy for stage III NSCLC who did not have disease progression after two or more cycles of platinum-based chemoradiotherapy | PACIFIC | PFS and OS | 713 | 2018 |
| Atezolizumab | Second line therapy of NSCLC | OAK | OS | 850 | 2016 |
| Pembrolizumab | Second line therapy foradvanced urothelial carcinoma | KEYNOTE-45 | PFS and OS | 542 | 2017 |
| Pembrolizumab | First-line NSCLC with tumors express PD-L1 > 50% as determined by an FDA-approved test | KEYNOTE-24 | PFS | 305 | 2017 |
| Pembrolizumab | Second line therapy of metastatic NSCLC whose tumors express PD-L1 | KEYNOTE-010 | PFS and OS | 1034 | 2016 |
| Pembrolizumab | First-line therapy of melanoma | KEYNOTE-006 | PFS and OS | 834 | 2015 |
| Nivolumab | Adjuvant therapy of melanoma | CHECKMATE-238 | RFS | 906 | 2017 |
| Nivolumab | Second line therapy of squamous-cell carcinoma of the head and neck | CHECKMATE-141 | OS | 361 | 2016 |
| Nivolumab | Second line therapy of renal cell carcinoma | CHECKMATE-025 | OS | 821 | 2015 |
| Nivolumab | Second line therapy of advanced squamous-cell NSCLC | CHECKMATE-017 | OS | 272 | 2015 |
| Nivolumab | Second line therapy of advanced nonsquamous NSCLC | CHECKMATE-057 | OS | 582 | 2015 |
| Nivolumab | First line therapy of BRAF wild-type unresectable or metastatic melanoma | CHECKMATE-066 | OS | 418 | 2015 |
| Nivolumab with or without ipilimumab | First line therapy of unresectable or metastatic melanoma | CHECKMATE-067 | PFS and OS | 945 | 2015 |
| Pembrolizumab | Second line therapy of unresectable or metastatic melanoma | KEYNOTE-002 | PFS | 540 | 2015 |
| Ipilimumab | Adjuvant therapy of melanoma | EORTC-18071 | RFS | 951 | 2015 |
| Ipilimumab | Second line therapy of unresectable or metastatic melanoma | MDX010–20 | OS | 676 | 2011 |
Clinical benefit of immune checkpoint inhibitors
Eighteen pivotal RCTs were included for the value assessment, with 5 trials had two experimental arms. By the ESMO-MCBS 1.1, for the 16 trials in the non-curative setting, 8 trials were grade five (the highest), and 8 trials grade four. For the two trials in the adjuvant setting, both were grade A. Thus, all trials met the ESMO-MCBS meaningful benefit threshold. Three trials met the ESMO-MCBS long term benefit criteria, all with the primary endpoint of PFS. Twelve of trials meet the criteria of improved toxicity (less grade 3–4 toxicities impacting on daily well-being) and only one trial was considered as increased toxic death.
For trials with ESMO-MCBS grade of 4, the median NHB was 49.3 (range 17.4–72.3), while for those with ESMO-MCBS grade of 5 or A, the median NHB was 56.8 (range 47.7–71.1).
ASCO VF Parameters for the Tail of the Curve Bonus Points
| Trial name | Evaluated endpoint | ESMO-MCBS | ASCO VF | ||||||||||||
| HR (95% CI) | QOL | Toxicity | Long term benefit | Score | Clinical benefit score | Toxicity score | Tail of the curve | Palliation | QOL | Treatment-free interval | NHB | ||||
| KEYNOTE-189 | OS | 0.49 (0.38–0.64) | Not reported | Not improved | Not qualified | 4 | 51 | −7.5 | 16 | 0 | 0 | 0 | 59.5 | ||
| CHECKMATE-214 | OS | 0.63 (99.8%CI, 0.44–0.89 | Improved | Improved | Not qualified | 5 | 37 | 4.9 | 0 | 0 | 10 | 0 | 51.9 | ||
| PACIFIC | PFS | 0.52(0.42 to 0.65) | Not reported | Not improved | > 10% improvement in PFS at 1 year | 4 | 38.4 | −6.7 | 16 | 0 | 0 | 0 | 47.7 | ||
| OAK | OS | 0.73 (0.62–0.87) | Not reported | Improved | Not qualified | 5 | 27 | 2.7 | 0 | 0 | 0 | 0 | 29.7 | ||
| KEYNOTE-45 | OS | 0.73 (0.59–0.91) | Not reported | Improved | Not qualified | 4 | 27 | 1 | 20 | 0 | 0 | 0 | 48 | ||
| KEYNOTE-24 | OS | 0.60 (0.41–0.89) | Not reported | Improved | Not qualified | 5 | 40 | 5.9 | 0 | 0 | 0 | 0 | 45.9 | ||
| KEYNOTE-010-1a | OS | 0.61 (0.49–0.75) | Not reported | Improved | Not qualified | 5 | 39 | 5.7 | 20 | 0 | 0 | 0 | 64.7 | ||
| KEYNOTE-010-2b | OS | 0.71 (0.58–0.88) | Not reported | Improved | Not qualified | 3 | 29 | 6.6 | 20 | 0 | 0 | 0 | 55.6 | ||
| KEYNOTE-006-1c | OS | 0.69 (0.52–0.90) | Not reported | Improved | Not qualified | 5 | 31 | 2.3 | 16 | 0 | 0 | 0 | 49.3 | ||
| KEYNOTE-006-2d | OS | 0.63 (0.47–0.83) | Not reported | Improved | Not qualified | 5 | 37 | 3.1 | 16 | 0 | 0 | 0 | 56.1 | ||
| CHECKMATE-238 | RFS | 0.65 (0.51–0.83) | Not Improved | Improved | Not qualified | A | 35 | 3.8 | 0 | 0 | 0 | 0 | 38.8 | ||
| CHECKMATE-141 | OS | 0.70 (0.52, 0.92) | Improved | Improved | Not qualified | 4 | 30 | 7.1 | 20 | 10 | 10 | 0 | 77.1 | ||
| CHECKMATE-025 | OS | 0.73 (98.5% CI, 0.57–0.93) | Improved | Improved | Not qualified | 5 | 27 | 6.8 | 0 | 0 | 10 | 0 | 43.8 | ||
| CHECKMATE-017 | OS | 0.59 (0.44–0.79) | Not reported | Improved | Not qualified | 5 | 41 | 11.3 | 20 | 0 | 0 | 0 | 72.3 | ||
| CHECKMATE-057 | OS | 0.73 (0.60–0.89) | Not reported | Improved | Not qualified | 5 | 27 | 8.3 | 20 | 0 | 0 | 0 | 55.3 | ||
| CHECKMATE-066 | OS | 0.42 (99.79% CI, 0.25–0.73) | Not Improved | Not Improved | Not qualified | 4 | 58 | 2.4 | 16 | 0 | 0 | 0 | 76.4 | ||
| CHECKMATE- 067-1e | PFS | 0.42 (99.5% CI, 0.31 to 0.57) | Not reported | Increased toxic death | > 10% improvement in PFS at 1 year | 3 | 46.4 | −5.1 | 16 | 0 | 0 | 0 | 57.3 | ||
| CHECKMATE-067-2f | PFS | 0.57 (99.5% CI, 0.43 to 0.76) | Not reported | Improved | > 10% improvement in PFS at 1 year | 4 | 34.4 | 5.5 | 16 | 0 | 0 | 0 | 55.9 | ||
| KEYNOTE-002-1g | PFS | 0.57 (0.45–0.73) | Not reported | Improved | > 10% improvement in PFS at 1 year | 4 | 34.4 | 7.3 | 16 | 0 | 0 | 0 | 57.7 | ||
| KEYNOTE-002-2h | PFS | 0.50 (0.39–0.64) | Not reported | Improved | > 10% improvement in PFS at 1 year | 4 | 40 | 5.9 | 16 | 0 | 0 | 0 | 61.9 | ||
| EORTC-18071 | RFS | 0.75 (0.64–0.90) | Not improved | Not improved | Not qualified | A | 25 | −7.6 | 0 | 0 | 0 | 0 | 17.4 | ||
| MDX010–20-1i | OS | 0.66 (0.51–0.87) | Not reported | Not improved | Not qualified | 4 | 34 | 1 | 20 | 0 | 0 | 0 | 55 | ||
| MDX010–20-2j | OS | 0.68 (0.55–0.85) | Not reported | No improved | Not qualified | 4 | 32 | −0.34 | 20 | 0 | 0 | 0 | 51.7 | ||
| Trial name | Endpoint | Drug | Cancer | ESMO-MCBS | ASCO VF | ||||||||||
| HR (95% CI) | QOL | Toxicity | Long term benefit | Score | Clinical benefit score | Toxicity score | Tail of the curve | Palliation | QOL | Treatment-free interval | NHB | ||||
| KEYNOTE-189* | OS | Pembrolizumab | NSCLC | 0.49 (0.38–0.64) | Not reported | Not improved | Not qualified | 4 | 51 | −7.5 | 16 | 0 | 0 | 0 | 59.5 |
| CHECKMATE-214* | OS | Nivolumab | RCC | 0.63 (99.8%CI, 0.44–0.89 | Improved | Improved | Not qualified | 5 | 37 | 4.9 | 0 | 0 | 10 | 0 | 51.9 |
| PACIFIC* | PFS | Durvalumab | NSCLC | 0.52(0.42 to 0.65) | Not reported | Not improved | > 10% improvement in PFS at 1 year | 4 | 38.4 | −6.7 | 16 | 0 | 0 | 0 | 47.7 |
| OAK | OS | Atezolizumab | NSCLC | 0.73 (0.62–0.87) | Not reported | Improved | Not qualified | 5 | 27 | 2.7 | 0 | 0 | 0 | 0 | 29.7 |
| KEYNOTE-45 | OS | Pembrolizumab | UC | 0.73 (0.59–0.91) | Not reported | Improved | Not qualified | 4 | 27 | 1 | 20 | 0 | 0 | 0 | 48 |
| KEYNOTE-24 | OS | Pembrolizumab | NSCLC | 0.60 (0.41–0.89) | Not reported | Improved | Not qualified | 5 | 40 | 5.9 | 0 | 0 | 0 | 0 | 45.9 |
| KEYNOTE-010-1a* | OS | Pembrolizumab | NSCLC | 0.61 (0.49–0.75) | Not reported | Improved | Not qualified | 5 | 39 | 5.7 | 20 | 0 | 0 | 0 | 64.7 |
| KEYNOTE-010-2b* | OS | Pembrolizumab | NSCLC | 0.71 (0.58–0.88) | Not reported | Improved | Not qualified | 3 | 29 | 6.6 | 20 | 0 | 0 | 0 | 55.6 |
| KEYNOTE-006-1c | OS | Pembrolizumab | Melanoma | 0.69 (0.52–0.90) | Not reported | Improved | Not qualified | 5 | 31 | 2.3 | 16 | 0 | 0 | 0 | 49.3 |
| KEYNOTE-006-2d | OS | Pembrolizumab | Melanoma | 0.63 (0.47–0.83) | Not reported | Improved | Not qualified | 5 | 37 | 3.1 | 16 | 0 | 0 | 0 | 56.1 |
| CHECKMATE-238 | RFS | Nivolumab | Melanoma | 0.65 (0.51–0.83) | Not Improved | Improved | Not qualified | A | 35 | 3.8 | 0 | 0 | 0 | 0 | 38.8 |
| CHECKMATE-141 | OS | Nivolumab | SCCHN | 0.70 (0.52, 0.92) | Improved | Improved | Not qualified | 4 | 30 | 7.1 | 20 | 10 | 10 | 0 | 77.1 |
| CHECKMATE-025* | OS | Nivolumab | RCC | 0.73 (98.5% CI, 0.57–0.93) | Improved | Improved | Not qualified | 5 | 27 | 6.8 | 0 | 0 | 10 | 0 | 43.8 |
| CHECKMATE-017 | OS | Nivolumab | NSCLC | 0.59 (0.44–0.79) | Not reported | Improved | Not qualified | 5 | 41 | 11.3 | 20 | 0 | 0 | 0 | 72.3 |
| CHECKMATE-057* | OS | Nivolumab | NSCLC | 0.73 (0.60–0.89) | Not reported | Improved | Not qualified | 5 | 27 | 8.3 | 20 | 0 | 0 | 0 | 55.3 |
| CHECKMATE-066* | OS | Nivolumab | RCC | 0.42 (99.79% CI, 0.25–0.73) | Not Improved | Not Improved | Not qualified | 4 | 58 | 2.4 | 16 | 0 | 0 | 0 | 76.4 |
| CHECKMATE- 067-1e* | PFS | Nivolumab | Melanoma | 0.42 (99.5% CI, 0.31 to 0.57) | Not reported | Increased toxic death | > 10% improvement in PFS at 1 year | 3 | 46.4 | −5.1 | 16 | 0 | 0 | 0 | 57.3 |
| CHECKMATE-067-2f* | PFS | Nivolumab | Melanoma | 0.57 (99.5% CI, 0.43 to 0.76) | Not reported | Improved | > 10% improvement in PFS at 1 year | 4 | 34.4 | 5.5 | 16 | 0 | 0 | 0 | 55.9 |
| KEYNOTE-002-1g | PFS | Pembrolizumab | Melanoma | 0.57 (0.45–0.73) | Not reported | Improved | > 10% improvement in PFS at 1 year | 4 | 34.4 | 7.3 | 16 | 0 | 0 | 0 | 57.7 |
| KEYNOTE-002-2h | PFS | Pembrolizumab | Melanoma | 0.50 (0.39–0.64) | Not reported | Improved | > 10% improvement in PFS at 1 year | 4 | 40 | 5.9 | 16 | 0 | 0 | 0 | 61.9 |
| EORTC-18071* | RFS | Ipilimumab | Melanoma | 0.75 (0.64–0.90) | Not improved | Not improved | Not qualified | A | 25 | −7.6 | 0 | 0 | 0 | 0 | 17.4 |
| MDX010–20-1i* | OS | Ipilimumab | Melanoma | 0.66 (0.51–0.87) | Not reported | Not improved | Not qualified | 4 | 34 | 1 | 20 | 0 | 0 | 0 | 55 |
| MDX010–20-2j* | OS | Ipilimumab | Melanoma | 0.68 (0.55–0.85) | Not reported | No improved | Not qualified | 4 | 32 | −0.34 | 20 | 0 | 0 | 0 | 51.7 |
Updated clinical benefit
Comparison of ASCO VFs evaluated based on initial reports and updated reports
| Trial name | Evaluated endpoint | ESMO-MCBS | ASCO VF | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| HR (95% CI) | QOL | Toxicity | Long term benefit | Score | Clinical benefit score | Toxicity score | Tail of the curve | Palliation | QOL | Treatment free interval | NHB | ||
| PACIFIC-Updateda | OS | 0.68 (99.73% CI, 0.47–0.997) | Not reported | Not improved | Not qualified | 4 | 32 | −6.2 | 16 | 0 | 0 | 0 | 41.8 |
| OAK-Updated | OS | 0.75 (0.64–0.89) | Not improved | Improved | Not qualified | 5 | 25 | 2.7 | 0 | 10 | 0 | 0 | 37.7 |
| KEYNOTE-45-Updated | OS | 0.73 (0.59–0.91) | Improved | Improved | Not qualified | 4 | 27 | 1 | 20 | 10 | 10 | 68 | |
| KEYNOTE-24-Updated | OS | 0.60 (0.41–0.89) | Improved | Improved | Not qualified | 5 | 40 | 5.9 | 0 | 10 | 10 | 0 | 65.9 |
| KEYNOTE-006-1-Updated | OS | 0.68 (0.53–0.87) | Improved | Not improved | Not qualified | 5 | 32 | 2.3 | 16 | 0 | 10 | 0 | 60.3 |
| KEYNOTE-006-2-Updated | OS | 0.68 (0.53–0.86) | Improved | Not improved | Not qualified | 5 | 32 | 3.1 | 16 | 0 | 10 | 0 | 61.1 |
| CHECKMATE-141-Updated | OS | 0.68 (0.54–0.86) | Improved | Improved | Not qualified | 5 | 32 | 7.1 | 20 | 10 | 10 | 0 | 79.1 |
| CHECKMATE-025-Updateda | OS | 0.73 (98.5% CI, 0.57–0.93) | Improved | Improved | Not qualified | 5 | 27 | 6.8 | 0 | 10 | 10 | 0 | 53.8 |
| CHECKMATE-017-Updated | OS | 0.62 (0.48–0.80) | Improved | Improved | Not qualified | 5 | 38 | 11.3 | 20 | 10 | 10 | 0 | 89.3 |
| CHECKMATE-057-Updateda | OS | 0.73 (0.62–0.88) | Improved | Improved | Not qualified | 5 | 27 | 8.3 | 20 | 10 | 10 | 0 | 75.3 |
| CHECKMATE-066-Updateda | OS | 0.42 (99.79% CI, 0.25–0.73) | Improved | No Improved | Not qualified | 5 | 58 | 2.4 | 16 | 0 | 10 | 0 | 86.4 |
| CHECKMATE-067-1-Updateda | OS | 0.54 (0.44–0.67) | Not Improved | Increased | Not qualified | 3 | 46 | −9.0 | 20 | 0 | 0 | 10 | 67 |
| CHECKMATE-067-2-Updateda | OS | 0.65 (0.53–0.79) | Not Improved | Not improved | Not qualified | 4 | 35 | 0.3 | 20 | 0 | 0 | 0 | 55.3 |
| KEYNOTE-002-1-Updated | PFS | 0.57 (0.45–0.73) | Improved | Improved | > 10% improvement in PFS at 1 year but without OS benefit | 4 | 34.4 | 5.4 | 16 | 10 | 10 | 0 | 75.8 |
| KEYNOTE-002-2-Updated | PFS | 0.50 (0.39–0.64) | Improved | Improved | > 10% improvement in PFS at 1 year but without OS benefit | 4 | 40 | 4.4 | 16 | 10 | 10 | 0 | 80.4 |
| EORTC-18071-Updateda | OS | 0.72 (0.58–0.88) | Not improved | Not improved | Not qualified | A | 28 | −8.5 | 0 | 0 | 0 | 0 | 19.5 |
| MDX010–20-1-Updateda | OS | 0.66 (0.51–0.87) | No improved | No improved | Not qualified | 4 | 34 | 1 | 20 | 0 | 0 | 0 | 55 |
| MDX010–20-2-Updateda | OS | 0.68 (0.55–0.85) | No improved | No improved | Not qualified | 4 | 32 | −0.3 | 20 | 0 | 0 | 0 | 51.7 |
Discussion
A previous study by Vivot A et al. [7], which assessed the clinical benefit of new drugs for treating advanced solid tumors aproved by the US FDA between 2000 and 2015 using ASCO-VF and ESMO-MCBS, reported that 13 (35%) out of 51 approved anticancer drugs showed a meaningful clinical benefit (scale levels 4 and 5) by ESMO-MCBS, and the median drug value was 37 (interquartile range 3.4–67) by ASCO-VF. Another study by Tibau A et al. [5] evaluated the magnitude of clinical benefit of cancer drugs approved by the US FDA from January 2006 to December 2016 using ESMO-MCBS, and found that fewer than half of RCTs supporting FDA approval meet the threshold for clinically meaningful benefit. However, less than 20% of the approved agents were immune checkpoint inhibitors in these studies, with more than 60% of approved agents being target therapy.
In our analysis, all trials met the ESMO-MCBS meaningful benefit threshold and by the ASCO-VF, the median NHB of drugs was 55.3 (range 17.4–77.1). Although caution should be taken in interpreting across study comparisons, due to the fact we used updated ASCO-VF and ESMO-MCBS, the clinical benefit seems greater in immune checkpoint inhibitors than other approved cancer drugs. Only two trials in the adjuvant setting were included in our study. Both trial meet the ESMO-MCBS 1.1 threshold for meaningful benefit. NHB of the two agents were 37.8 and 17.4, which seems lower than those in the metastatic setting. Further studies are need to evaluate whether the clinical benefit of immune checkpoint inhibitors in the adjuvant setting is consistent with those in the metastatic setting with more agents were approved in the adjuvant setting.
Recently, Ben-Aharon et al. [23], tried to determine whether immuno-oncology agents approved by the FDA fulfill the durable survival threshold defined in the updated ASCO-VF. They found only 3 drug indications fulfilled the threshold. However, in our study, 12 of 18 approved indications gained the bonus points for durable survival benefits. Several issues may explain the discrepancies. First,, as pointed by Vivot et al. in their letter [24] to the editor and Schnipper et al. in their commentary [25], Ben-Aharon et al. used raw proportions of patients at risk (ie, number of patients still at risk divided by the number of patients randomized) to estimate the survival proportion instead of using the probability displayed on Kaplan-Meier curves, which may have may have disqualify trials that actually met the ASCO-VF criteria for long term benefit.. Second, only 10 indications approved based on RCTs were eligible for their analysis. Only recently more RCTs of immunotherapy have been finished and reported. And they were never evaluated with ESMO-MCBS. Our study provided important and comprehensive evaluation of approved immune checkpoint inhibitors in RCTs.
Although our study did not aim to or was powered to assess the consistency of updated ASCO-VF and the ESMO-MCBS 1.1 due to limited number of RCTs included, Clinical benefits by updated ASCO-VF and the ESMO-MCBS 1.1 yielded some sorts of consistencies. For trials with ESMO-MCBS grade of 5 or A, the median NHB was numerical higher than those with ESMO-MCBS grade of 4. A recent study [26] that evaluated the concordance between the two frameworks in the noncurative setting showed that agreement between the frameworks was higher than observed in other studies that sought to compare them [27, 28]. This study was done by the authorship group of the two frameworks (vs independent groups). Concordance will likely be greater when those individuals who created the value frameworks are the ones scoring/grading. Another cohort comparing the two frameworks has also drawn similar conclusions [29]. The issue of framework utility in the general oncology community has been raised recently [30].
We found that 12 of 18 indications gained the bonus points for tails of the curve, while only 3 indications met the ESMO-MCBS long term benefit criteria, all with the primary endpoint of PFS. This discrepancy is not surprising given the differences in their criteria. To qualify for the long-term plateau by ESMO-MCBS 1.1 [4], overall survival advantage need to be observed at 5 years if the median overall survival in the standard arm ≤12 months. Currently none of these trials in the non-curative setting reported survival results at 5 years.
When updated results were incorporated in the assessment, clinical benefit of most approved immune checkpoint inhibitors increased, largely due to the statistically significant improvement in the QoL or/and cancer-related symptoms that were not available in the primary reports but reported subsequently. Thus, the score may change when data mature. Our results emphasized the importance of PROs in accurately evaluating the clinical benefit of immune checkpoint inhibitors.
Our study has several limitations. First, toxicities information were extracted from published articles, which often reported only adverse events that occurred in at least 10% of the treated patients, thus, the toxicity grade by ASCO VF may change with complete toxicity information. Second, although we conducted comprehensive research, PROs reports were not available for all approved agents, clinical benefit of these agents may change when PROs report were available. Third, we focused on clinical benefit of immune checkpoint inhibitors, and no comparisons to approved chemotherapy or other agents over a similar time period were conducted.
Conclusion
In summary, all of the approved immune checkpoint inhibitors based on RCTs meet the ESMO-MCBS threshold for clinical benefit, and two thirds of these approved agents fulfilled the durable benefit thresholds in the updated ASCO VF. This information may be used in future analysis to better define clinical benefits of immunotherapies.