Original Investigation
March 25, 2024
Effectiveness of an Artificial Intelligence–Enabled Intervention for Detecting Clinical Deterioration
Robert J. Gallo, Lisa Shieh, Margaret Smith, et al
JAMA Intern Med. Published online March 25, 2024. doi:10.1001/jamainternmed.2024.0084
Question Is an artificial intelligence (AI) deterioration model–enabled intervention associated with a decreased risk of escalations in care during hospitalization?
Findings In this cohort study of 9938 patients hospitalized at a single academic center in 2021 and 2022, exposure to the intervention was associated with a 10.4–percentage point absolute risk reduction in the primary composite outcome of rapid response team activation, transfer to the intensive care unit, or cardiopulmonary arrest during hospitalization.
Meaning Findings of this study suggest that use of an AI deterioration model–enabled intervention was associated with a decreased risk of escalations in care during hospitalization.
Abstract
Importance Inpatient clinical deterioration is associated with substantial morbidity and mortality but may be easily missed by clinicians. Early warning scores have been developed to alert clinicians to patients at high risk of clinical deterioration, but there is limited evidence for their effectiveness.
Objective To evaluate the effectiveness of an artificial intelligence deterioration model–enabled intervention to reduce the risk of escalations in care among hospitalized patients using a study design that facilitates stronger causal inference.
Design, Setting, and Participants This cohort study used a regression discontinuity design that controlled for confounding and was based on Epic Deterioration Index (EDI; Epic Systems Corporation) prediction model scores. Compared with other observational research, the regression discontinuity design facilitates causal analysis. Hospitalized adults were included from 4 general internal medicine units in 1 academic hospital from January 17, 2021, through November 16, 2022.
Exposure An artificial intelligence deterioration model–enabled intervention, consisting of alerts based on an EDI score threshold with an associated collaborative workflow among nurses and physicians.
Main Outcomes and Measures The primary outcome was escalations in care, including rapid response team activation, transfer to the intensive care unit, or cardiopulmonary arrest during hospitalization.
Results During the study, 9938 patients were admitted to 1 of the 4 units, with 963 patients (median [IQR] age, 76.1 [64.2-86.2] years; 498 males [52.3%]) included within the primary regression discontinuity analysis. The median (IQR) Elixhauser Comorbidity Index score in the primary analysis cohort was 10 (0-24). The intervention was associated with a −10.4–percentage point (95% CI, −20.1 to −0.8 percentage points; P = .03) absolute risk reduction in the primary outcome for patients at the EDI score threshold. There was no evidence of a discontinuity in measured confounders at the EDI score threshold.
Conclusions and Relevance Using a regression discontinuity design, this cohort study found that the implementation of an artificial intelligence deterioration model–enabled intervention was associated with a significantly decreased risk of escalations in care among inpatients. These results provide evidence for the effectiveness of this intervention and support its further expansion and testing in other care settings.
