Validation of the STOP-Bang questionnaire as a preoperative screening tool for obstructive sleep apnea: a systematic review and meta-analysis

The protocol of this study was registered in the International Prospective Register of Systematic Reviews (PROSPERO; registration CRD42021260451). The study was completed in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) guideline [20].

The literature search was performed by an information specialist (ME) using the Ovid platform for the following databases: MedlineALL, Embase, Cochrane Central Register of Controlled Trials, Cochrane Database of Systematic Reviews, APA PsycINFO, and Journals@Ovid. CINAHL, the Web of Science, and Scopus (Elsevier) were also searched. Search components consisted of (“stop-bang” or “stopbang”) AND (perioperative or postoperative or surgery) related terms. Searches were limited to the years 2008 (development of STOP-Bang questionnaire) to May 14, 2021. No other limits were applied. Literature surveillance was performed through November 2021. The Medline search strategy is provided in the supplemental material (Appendix). 1

Title and abstract screening, and full text evaluation were independently completed by two reviewers (MH, NG) using Covidence. Full text articles meeting the following inclusion criteria were included: 1) the study screened for OSA using the STOP-Bang questionnaire in adult patients aged ≥18 years undergoing surgery; 2) OSA diagnosis confirmed by PSG or home sleep apnea testing (HSAT); 3) severity of OSA measured by Apnea Hypopnea Index (AHI) or Respiratory Disturbance Index (RDI) cut-offs ≥5, ≥ 15, and ≥ 30 events per hour; and 4) accuracy of the STOP-Bang questionnaire assessed with predictive parameters. The two reviewers extracted data from the included studies with a standardized form. A third reviewer (AS) resolved any discrepancy between the reviewers. Data collection was managed in Excel (Redmond, United States).

Two reviewers (MH, NG) independently evaluated bias of the included studies. The assessment was conducted using criteria for internal and external validity coded using the Cochrane Screening and Diagnostic Tests Methods Group [21]. The result of the evaluation was compared and a third reviewer (AS) resolved any discrepancies. Internal validity was assessed using the following criteria: valid reference standard, definition of disease, blind execution of index and reference tests, interpretation of index test independent of clinical information, and study design. External validity was assessed using the following criteria: disease spectrum, clinical setting, previous screening or referral filter, demographic information, explicit cut-off of index test, percentage of missing participants, management of missing data, and selection of participants for the reference test. Furthermore, the Quality Assessment of Diagnostic Accuracy Studies (QUADAS) tool was used by the reviewers to rate the quality of individual included study on a scale ranging from 0 to 14 [22].

Meta-analysis was performed with Review Manager Version 5.4 and Meta-disc V.1.4. For each of the included studies, 2 × 2 contingency tables were created to obtain predictive parameters with 95% confidence intervals. The following pooled predictive parameters were calculated using a bivariate random-effects model: sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), diagnostic odds ratio (DOR), likelihood ratios, and area under the curve (AUC) to evaluate the validity of the STOP-Bang questionnaire for different OSA severities defined by AHI cut-offs: AHI ≥ 5 (all), AHI ≥ 15 (moderate-to-severe), and AHI ≥ 30 (severe) events per hour. A STOP-Bang score of three or greater was accepted as the threshold and post-test probabilities were calculated as described by Brooks et al. [23] Also, the pooled predictive parameters of additional STOP-Bang score thresholds were calculated for different OSA severities.

Meta-regression and sensitivity analysis were performed for moderate-to-severe and severe OSA using Open Meta Analyst software [24] for categorical variables (validation tools and study type) and continuous variables (age, male gender, BMI, neck circumference, prevalence, and sample size). We focused to measure the association between these variables and the combined estimates of sensitivity, specificity, and log scale diagnostic odds ratio. Leave-one-study-out analysis was performed to examine the effect, if any, of individual study on the reliability of the combined estimates. Level of statistical significance was set at p value < 0.05.

The included studies had QUADAS scores ranging from 9 to 13, denoting a moderate risk of bias (Table 1). PSG or HSAT was used as a reference test in all included studies to determine the accuracy of the STOP-Bang questionnaire (Table 2). Three studies [42, 43, 45] exclusively used PSG, two [41, 48] used a mix of PSG and HSAT, and five [44, 46, 47, 49, 50] used HSAT. Although the standard for the diagnosis of OSA is PSG, there were no significant disparities between studies that used PSG or HSAT regarding prevalence of OSA (Table 2) and accuracy of the STOP-Bang questionnaire (Fig. 2).

The evaluation of internal and external validities of the included studies is summarized in Supplementary Tables S2 and S3. Regarding the internal validity, index and reference tests were blindly executed in five studies [41, 44, 45, 49, 50] and STOP-Bang scores were interpreted independently from clinical data in two studies [44, 50] (Supplementary Table S2). All but one study [43], which had unspecified inclusion criteria, fully described their inclusion and exclusion criteria. In all 10 studies, adequate information was provided to describe the study setting and the demographics of the surgical patients, including age, sex, and BMI. Although one study [43] did not randomly select patients for PSG, all studies applied the STOP-Bang questionnaire without pre-screening for OSA. Overall, there is a low risk of bias in subject selection for the reference test across the 10 studies.

For the STOP-Bang questionnaire, the pooled predictive parameters of a score three or greater to screen for OSA in patients undergoing surgery are presented in Table 3 and Figs. 2 and 3. The prevalence of all, moderate-to-severe, and severe OSA was 65, 38, and 17% respectively. The pooled sensitivity of the STOP-Bang questionnaire was high at 85% (95%CI: 82, 88%, I²: 40.9%), 88% (95%CI: 85, 89%, I²: 85.7%), and 90% (95%CI: 87, 93%, I²: 86.9%) in screening for all, moderate-to-severe, and severe OSA, respectively. The pooled specificity was moderate at 47% (95%CI: 42, 52%, I²: 87.2%), 29% (95%CI: 27, 32%, I²: 89.9%), and 27% (95%CI: 25, 29%, I²: 93.1%) for all, moderate-to-severe, and severe OSA, respectively.

The pooled positive predictive value (PPV) was highest at 75% (95%CI: 71.8, 77.7%) in detecting all OSA, and the corresponding PPVs for moderate-to-severe and severe OSA were 43% (95%CI: 40.8, 45.0%) and 20% (95%CI: 18.5, 22.2%), respectively (Table 3). The negative predictive value (NPV) for severe OSA was highest at 93.2% (95%CI: 90.9, 95.1%), indicating that the STOP-Bang questionnaire can reasonably rule-out severe OSA. A negative score of 0–2 would decrease the probability of diagnosing severe OSA from 17.0 to 5.2%. The corresponding NPV values for all and moderate-to-severe OSA were 62.7% (95% CI: 56.9, 68.1%) and 79.6% (95% CI: 76.2, 82.6). The area under the curve (AUC) was 0.84, 0.67, and 0.63 for all, moderate-to-severe, and severe OSA, respectively.

The accuracy of different STOP-Bang score cut-offs in surgical patients for all OSA (n = 5722), moderate-to-severe OSA (n = 12,207) and severe OSA (n = 9878) are summarized in Supplementary Table S4. With the increase in the STOP-Bang threshold from 3 to 5, the sensitivity diminished from 88 to 50% for moderate-to-severe OSA and from 90 to 61% for severe OSA. As well, there was an increase in specificity from 29 to 78% for moderate-to-severe and from 27 to 75% for severe OSA. The PPV was highest at 86% with a STOP-Bang threshold of 6 or greater for detecting all OSA. Similarly, the NPV was highest for severe OSA at 94% for a threshold of 4 or greater.

Meta-regression and sensitivity analysis were conducted for moderate-to-severe OSA with 12 study groups and for severe OSA in eight studies (Supplementary Tables Sand S). The analysis revealed that continuous variables marginally altered the combined estimates without significant effect on the results. Similarly, the categorical variables also marginally altered the combined estimates without significance. There was no significant effect on the results by any individual study as shown by leave-one-study-out analysis (Supplementary Fig. S). 5 6 1

Despite rising awareness and increase in prevalence of OSA in patients undergoing surgery, [53, 57] the vast majority of patients with OSA are unidentified preoperatively [2, 3, 52]. Undiagnosed OSA has been associated with difficult airway management [58] and increased postoperative complications including cardiovascular events, reintubation, respiratory complications, and longer hospital stay [2, 17, 59–61]. Notably, preoperative use of the STOP-Bang questionnaire to screen surgical patients to detect undiagnosed OSA has been shown to predict postoperative complications [17, 18, 28, 59, 62]. Of note, several of these studies are non-randomized, observational studies [60–62].

An increased severity of OSA may be associated with an increased rate of postoperative complications. Severe OSA was found to be associated with increased risk of postoperative cardiac complications [2]. Similarly, a higher incidence of postoperative complications was associated with higher OSA severity [63]. As higher STOP-Bang score is associated with higher risk of moderate-to-severe and severe OSA, [41] our findings indicate that the STOP-Bang questionnaire is a valid screening tool for preoperative risk stratification.

A recent study found that patients identified by the STOP-Bang questionnaire (score ≥ 3) as at increased risk for OSA had a 4-fold increase in post-operative cardiopulmonary events [59]. Similarly, patients with STOP-Bang score ≥ 3 experienced worse perioperative respiratory outcomes and prolonged hospital stay [62]. As such, missed awareness of OSA in surgical patients can put substantial strain on the healthcare system due to increased consumption of resources in the form of intensive care, increased ventilator support, and longer length of hospitalization [64]. Patients with OSA and compliant with their continuous positive airway pressure (CPAP) therapy were shown to have improved oxygen desaturation index on the night of surgery and were less likely to require oxygen therapy [65]. In addition, surgical patients with OSA and a CPAP prescription were associated with fewer cardiovascular complications, [17] further highlighting the importance of preoperative identification of undiagnosed OSA.

Nevertheless, limited time between preoperative evaluation and surgery, patient hesitance to undergo sleep testing, and long waitlists for sleep clinics are barriers to recognizing undiagnosed OSA. This underscores the importance of access to a robust and easy-to-administer screening tool with a high predictive accuracy. We report that that the STOP-Bang questionnaire is a valid screening tool that addresses this need with a high AUC of 0.84 for clinicians to risk-stratify preoperative patients and to plan mitigation for perioperative complications associated with OSA. Surgical patients at high risk of OSA should be considered for postoperative monitoring such as continuous oximetry and capnography [66, 67].

There are some limitations in our study. First, both PSG and HSAT were used as diagnostic tools for OSA in the included studies. Although the two are often equitable, some heterogeneity may be present as PSG is the diagnostic standard. Secondly, the internal validity was difficult to assess as blinding of the index and reference tests was unclear. Nevertheless, QUADAS tool was used to provide additional evaluation of the quality of the included studies. Lastly, our study population included a variety of surgical procedures, which may limit the applicability of our results to specific surgical populations. The combination of methodological variations in the diagnostic tools, the variability in prevalence of OSA across the studies, and the various surgical procedures likely resulted in high heterogeneity of the predictive parameters. In anticipation of this heterogeneity, a random-effects model was used for the meta-analysis. Nevertheless, our study presents a current review of the literature on the accuracy of the STOP-Bang questionnaire as a preoperative screening tool in the surgical population.

Not applicable.

Not applicable.

Frances Chung reports research support from the University Health Network Foundation, ResMed Foundation, Ontario Ministry of Health Innovation Grant, consultant to Takeda Pharma, Up-to-date royalties, STOP-Bang proprietary to University Health Network. Mahesh Nagappa reports grants from the Academic Medical Organization of South-western Ontario. All other authors have no conflicts of interest to disclose.

All data generated or analysed during this study are included in this published article and its supplementary information files.