Original Investigation
August 23, 2023
Risk-Adjusted Cumulative Sum for Early Detection of Hospitals With Excess Perioperative Mortality
Vivi W. Chen, Alexis P. Chidi, Yongquan Dong, et al
JAMA Surg. 2023;158(11):1176-1183. doi:10.1001/jamasurg.2023.3673
Question Is use of the risk-adjusted cumulative sum (CUSUM) associated with improvement in early detection of hospitals with excess perioperative mortality relative to episodic performance evaluation?
Findings In this national, hospital-level, comparative effectiveness study of 697 566 patients undergoing surgery across 104 Veterans Affairs hospitals, the CUSUM identified hospitals with excess perioperative mortality significantly earlier than episodic evaluation and was associated with future performance.
Meaning Findings suggest that the CUSUM represents a useful tool that could be implemented within current national quality improvement programs and would likely enhance both quality and performance improvement efforts.
Abstract
Importance National surgical quality improvement programs lack tools for early detection of quality or safety concerns, which risks patient safety because of delayed recognition of poor performance.
Objective To compare the risk-adjusted cumulative sum (CUSUM) with episodic evaluation for early detection of hospitals with excess perioperative mortality.
Design, Setting, and Participants National, observational, hospital-level, comparative effectiveness study of 697 566 patients. Identification of hospitals with excess, risk-adjusted, quarterly 30-day mortality using observed to expected ratios (ie, current criterion standard in the Veterans Affairs Surgical Quality Improvement Program) was compared with the risk-adjusted CUSUM. Patients included in the study underwent a noncardiac operation at a Veterans Affairs hospital, had a record in the Veterans Affairs Surgical Quality Improvement Program (January 1, 2011, through December 31, 2016), and were aged 18 years or older.
Main Outcome and Measure Number of hospitals identified as having excess risk-adjusted 30-day mortality.
Results The cohort included 697 566 patients treated at 104 hospitals across 24 quarters. The mean (SD) age was 60.9 (13.2) years, 91.4% were male, and 8.6% were female. For each hospital, the median number of quarters detected with observed to expected ratios, at least 1 CUSUM signal, and more than 1 CUSUM signal was 2 quarters (IQR, 1-4 quarters), 8 quarters (IQR, 4-11 quarters), and 3 quarters (IQR, 1-4 quarters), respectively. During 2496 total quarters of data, outlier hospitals were identified 33.3% of the time (830 quarters) with at least 1 CUSUM signal within a quarter, 12.5% (311 quarters) with more than 1 CUSUM signal, and 11.0% (274 quarters) with observed to expected ratios at the end of the quarter. The CUSUM detection occurred a median of 49 days (IQR, 25-63 days) before observed to expected ratio reporting (1 signal, 35 days [IQR, 17-54 days]; 2 signals, 49 days [IQR, 26-61 days]; 3 signals, 58 days [IQR, 44-69 days]; ≥4 signals, 49 days [IQR, 42-69 days]; trend test, P < .001). Of 274 hospital quarters detected with observed to expected ratios, 72.6% (199) were concurrently detected by at least 1 CUSUM signal vs 42.7% (117) by more than 1 CUSUM signal. There was a dose-response relationship between the number of CUSUM signals in a quarter and the median observed to expected ratio (0 signals, 0.63; 1 signal, 1.28; 2 signals, 1.58; 3 signals, 2.08; ≥4 signals, 2.49; trend test, P < .001).
Conclusions This study found that with CUSUM, hospitals with excess perioperative mortality can be identified well in advance of standard end-of-quarter reporting, which suggests episodic evaluation strategies fail to detect out-of-control processes and place patients at risk. Continuous performance evaluation tools should be adopted in national quality improvement programs to prevent avoidable patient harm.
