Original Investigation 

Association of VA Surgeons

Expectations vs Reality of an Intraoperative Artificial Intelligence Intervention

Melissa Thornton, Benjamin A. Y. Cher, Cameron Macdonald, et al

JAMA Surg Published Online: January 14, 2026

doi: 10.1001/jamasurg.2025.6029

Key Points

Question  How can we develop a more effective approach for implementing artificial intelligence (AI)–based interventions that improve intra- and postoperative care?

Findings  In this multisite qualitative implementation study of AI-based Operating Room Black Box technology, the following 4 major gaps between the program’s promise and delivery that could impair implementation were identified: (1) need for additional AI training, (2) difficulty accessing data, (3) limited prediction of postoperative complications, and (4) lack of academic deliverables.

Meaning  There are significant gaps between the expectations and reality of this AI-based intervention that must be addressed for successful implementation.

Abstract

Importance  Having significant gaps between the expectations and reality of artificial intelligence–based programs can be a major barrier to successful implementation. This is the first multisite implementation assessment of gaps between surgeon expectations and real-world effects of the Operating Room Black Box, a novel intervention that leverages artificial intelligence to improve surgical outcomes.

Objective  To identify barriers and facilitators to implementing artificial intelligence–based interventions that improve intra- and postoperative care.

Design, Setting, and Participants  This qualitative study was conducted at 3 large academic centers via semistructured interviews with surgeons and implementation leaders of the AI intervention to identify areas where expectations of the technology misaligned with their experiences. Thirty surgeons and 17 implementation leaders from 3 centers that implemented the AI intervention were interviewed. Data were collected and analyzed between 2021 and 2024.

Exposure  Implementation of the AI intervention.

Main Outcomes and Measures  The primary outcome was areas of misalignment between participant expectations of the AI intervention technology and actual program deliverables.

Results  Of 30 surgeons and 17 implementation leaders interviewed, most surgeons (17 [57%]) were between the ages of 35 and 50 years, and implementation leaders were older, typically between 51 and 80 years old (6 [35%]). Eight surgeons (27%) and 4 implementation leaders (24%) were female. Most surgeons (17 [57%]) had neutral views of the technology, 11 (37%) expressed positive views, and 2 (7%) had negative views. Interviewees identified the following 4 major themes that highlighted misalignment between user expectations and the experience of using the technology: (1) the artificial intelligence model needed considerable additional training to be usable; (2) accessing data on surgical cases was difficult and time consuming; (3) the program showed limited ability to predict postoperative complications; and (4) the program generated few academic deliverables.

Conclusions and Relevance  Per the results of this multisite qualitative study, successfully implementing interventions based on artificial intelligence may require deliberate efforts to minimize gaps between what surgeons expect from the interventions and what they can deliver. Our evaluation of this study’s AI intervention offers lessons for addressing this critical barrier to implementation.