Choosing PD-1 Inhibitors in Cancer Treatment: Insights from ASCO and ESMO Value Scores

Phase III RCTs that compared nivolumab and pembrolizumab alone or in combination with chemotherapy, hormonal therapy, other targeted agents, etc., to the same regimen without them used in the intervention group irrespective of the cancer type and stage were identified.

Reports of secondary, subset, or pooled data, phase I or II trials, animal studies, or trials that assessed drug delivery or single-drug dosing schedules were excluded.

Systematic search of electronic databases including the Cochrane Controlled Trials Register on the Cochrane Library, MEDLINE, EMBASE, and Science Citation Index was conducted using the terms nivolumab, pembrolizumab and PD-1 inhibitor, with a cutoff date of Nov 30, 2019. Both MeSH and free text terms were used to identify relevant articles. Reference lists of pertinent retrieved articles were reviewed for additional studies, and ClinicalTrials.gov↗ was also checked in June 2019 to ensure that data from previously published trials were updated on the registry.

Two authors (HW and HL) independently conducted the literature search, screened titles and abstracts for potential eligibility and full texts for final eligibility. In case of disagreement, a consensus was reached through discussion.

Treatments were classified as curative or palliative according to the trial population. Two reviewers extracted the data using a standardized extraction form, including but not limited to the trial name, phase, cancer type, PD-1 inhibitor used, dosing schedule, follow-up time and outcomes in accordance with the ASCO-VF and ESMO-MCBS for all eligible studies. ASCO-VF scores and ESMO-MCBS grades were independently recorded. Any discrepancies were discussed among all authors to establish a final score or grade.

To assess the monthly cost of therapeutic regimen including the cost of all anticancer drugs in the study regimen, we used the price for branded and generic drugs recorded in the Hospital Information System (HIS). Monthly costs were calculated over an average of 30 days based on the dosage schedule in all eligible trials for a patient weighing 60 kg with a body surface area of 1.70 m². Ultimately, incremental monthly drug costs as the difference between the experimental and control groups were reported. The most expensive one was recorded if there were several options of therapeutic regimen in the control group. All therapeutic regimens were adjusted to provide the price per 4-week period.

All data were collected using structured Excel sheets designed for this study. Statistical analysis was performed using IBM SPSS 25.0. Continuous data of ASCO-VF scores were plotted and analyzed to assess the normality of the underlying distribution. Since ASCO-VF has no explicit definition for what score is deemed “meaningful value achieved”; we split scores at the 75th percentile of ASCO-VF scores as the cutoff score, referring to the meaningful value achieved of ESMO-MCBS as a grade of 4, 5, B, or A. We split the cutoff scores for subsequent analyses. A score above the cutoff was defined as “meaningful value achieved” while a score below the cutoff indicated “meaningful value not achieved”. Receiver operating characteristic (ROC) curves were generated to establish the predictive value of ASCO-VF score to meet ESMO-MCBS grade and discriminate the agreement of these two value assessment tools. Subgroups analyses were performed according to palliative and curative intent of the eligible trials. Spearman’s correlation was used to assess the association between monthly cost and ASCO-VF score/ESMO-MCBS grade. A p < 0.05 was deemed significant for all analyses.

Of the 11,414 reports identified through search of electronic databases, 19 phase III RCTs eventually met our eligibility criteria (Borghaei et al., 2015; Brahmer et al., 2015; Motzer et al., 2015; Robert et al., 2015; Weber et al., 2015; Cella et al., 2016; Ferris et al., 2016; Herbst et al., 2016; Long et al., 2016; Reck et al., 2016; Bellmunt et al., 2017; Brahmer et al., 2017; Carbone et al., 2017; Harrington et al., 2017; Kang et al., 2017; Weber et al., 2017; Eggermont et al., 2018; Hellmann et al., 2018; Hodi et al., 2018; Reck et al., 2018; Reck et al., 2018; Shitara et al., 2018; Cohen et al., 2019; Mok et al., 2019; Wu et al., 2019). Of these, seven (36.8%) trials included treatments with nivolumab while 12 (63.2%) with pembrolizumab; eight (42.1%) trials involved treatments for non-small-cell lung cancer (NSCLC), five (26.3%) for melanoma, two (10.5%) for head and neck squamous cell carcinoma and gastric or gastro-esophageal junction cancer, respectively, and one each (5.3%) for urothelial cancer and renal-cell carcinoma, respectively. The longest follow-up time was 48 months of nivolumab for melanoma. The largest sample size was 1,537 of nivolumab for NSCLC (Table 1).

ASCO-VF scores ranged from 7 to 94.7 (Figure 1), and the scores were normally distributed. Median ASCO-VF score was 40.90, with inter-quartile range (IQR) 33.30. Ten (52.6%) trials fell below, while nine (50.1%) trials were above. Since ASCO-VF has no explicit definition of what score is deemed “meaningful value achieved”; we split scores at the 75th percentile of ASCO-VF scores—63.3 as the cutoff score, referring to the meaningful value achieved of ESMO-MCBS as a grade of 4, 5, B, or A. Therefore, five (26.3%) trials were above the threshold whereas 14 (73.3%) fell below. Eleven (57.9%) of the 19 RCTs met the ESMO-MCBS criteria for meaningful value achieved. Of the 14 trials that did not meet the ASCO-VF cutoff score, only eight did not meet the ESMO-MCBS “meaningful value achieved” criteria.

ROC curve was used to establish a discrimination threshold of ASCO-VF scores in relation to the ESMO-MCBS criteria, and the threshold score was approximately 40, which was comparatively close to the median ASCO-VF value scores. Nevertheless, the area under the curve was 0.795 (p＜0.05) (Figure 2), suggesting only fair predictive value. Agreement between ASCO-VF and ESMO-MCBS thresholds was only fair (κ = 0.412, p＜0.05).

Incremental monthly cost data, ranged from ¥28,426.40 to ¥47,661.88, were not normally distributed and were analyzed with non-parametric statistics. Incremental monthly cost and ASCO-VF scores were negatively correlated (Spearman’s ρ= −0.272; p = 0.260), and a negative correlation was also noted between incremental monthly cost and ESMO-MCBS grades (Spearman’s ρ= −0.088; p = 0.720).

To the best of our knowledge, this was the first study that applied ASCO-VF and ESMO-MCBS to comprehensively address the value of two PD-1 inhibitors, nivolumab and pembrolizumab, and offer insight into the association between these two frameworks.

Most studies focused on NSCLS and melanoma. In addition to NSCLC, melanoma, head and neck squamous cell carcinoma and gastric or gastro-esophageal junction cancer, only one trial compared nivolumab in renal cell carcinoma while another compared pembrolizumab in urothelial cancer (Cella et al., 2016).

There were some conflicting results. For gastric or gastro-esophageal junction cancer, ASCO-VF scores and ESMO-MCBS grades showed that both nivolumab and pembrolizumab were of little value, with ASCO-VF scores 7 vs. ESMO-MCBS grade 1 for nivolumab (Reck et al., 2016) and ASCO-VF scores 30 vs. ESMO-MCBS grade 3 for pembrolizumab (Eggermont et al., 2018). For the second-line treatment for recurrent or metastatic head and neck squamous cell carcinoma, nivolumab (ASCO-VF scores 82.5 vs. ESMO-MCBS grade 4) was valuable (Borghaei et al., 2015; Harrington et al., 2017), but pembrolizumab was not (ASCO-VF scores 30.8 vs. ESMO-MCBS grade 3) (Wu et al., 2019). For melanoma, discordant ASCO-VF scores and ESMO-MCBS grades were generated except CheckMate 066 (Robert et al., 2015; Long et al., 2016) and CheckMate 037 (Robert et al., 2015) examining the efficacy and safety nivolumab for melanoma. For CheckMate 066, ASCO-VF score was highest and ESMO-MCBS grade was only 4. For NSCLC, results were almost the same with ASCO-VF scores and ESMO-MCBS grades. For CheckMate 017 (Motzer et al., 2015; Reck et al., 2018) examining the second-line treatment of nivolumab for squamous-cell stage IIIB or IV NSCLC after platinum-based therapy, and KEYNOTE-024 (Herbst et al., 2016; Brahmer et al., 2017) examining the first-line pembrolizumab treatment for stage IV NSCLC with EGFR and ALK wild type, both ASCO-VF scores and ESMO-MCBS grades were high, demonstrating that they were valuable.

This study fills a crucial knowledge gap regarding the value of PD-1 inhibitors among cancer patients. Several strengths should be noted. Firstly, each trial had a wide range of ASCO-VF scores. Secondly, not all trials met the ESMO-MCBS “meaningful value achieved” criteria. Thirdly, only a fair association was found between ASCO-VF and ESMO-MCBS. Fourth, there was no correlation between incremental monthly cost and ASCO-VF scores or ESMO-MCBS grades.

Irrespective of whether the trials described patients with specific PD-1 expression level, high ASCO-VF scores and ESMO-MCBS grades were recorded.

Nevertheless, this study also had several limitations. Firstly, the ASCO-VF score and ESMO-MCBS grade of the trials with the largest sample size was not high and did not meet the ASCO-VF cutoff score and the ESMO-MCBS “meaningful value achieved” criteria. Secondly, the association between ASCO-VF and ESMO-MCBS was only moderate in a cohort of 19 trials, substantially similar to those reported by others except one study including only a small number of trials (n = 5). Extensive efforts are needed to improve convergence of the two value assessment tools based on our findings. Thirdly, since ASCO-VF has no explicit definition of what score is deemed “meaningful value achieved”; we split scores at the 75th percentile of ASCO-VF scores—63.3 as the cutoff score, referring to the meaningful value achieved of ESMO-MCBS as a grade of 4, 5, B, and A. Nevertheless, changing this cutoff score will change the degree of correlation between these two tools.

This study sheds light on the important clinical issue about the comparative value of PD-1-related treatment. Our results demonstrated the value of a drug should not be judged solely by its price. It is necessary to conduct value assessment to insight the date beyond RCTs.