{"id":29957,"date":"2026-03-11T04:20:00","date_gmt":"2026-03-10T20:20:00","guid":{"rendered":"https:\/\/csccm.org.cn\/?p=29957"},"modified":"2026-03-11T05:54:39","modified_gmt":"2026-03-10T21:54:39","slug":"chest%e5%8f%91%e8%a1%a8%e8%ae%ba%e6%96%87%ef%bc%9a%e4%b8%93%e5%ae%b6%e4%b8%8e%e5%a4%a7%e8%af%ad%e8%a8%80%e6%a8%a1%e5%9e%8b%e7%94%9f%e6%88%90%e7%9a%84%e6%9c%ba%e6%a2%b0%e9%80%9a%e6%b0%94%e9%a2%86","status":"publish","type":"post","link":"https:\/\/csccm.org.cn\/?p=29957","title":{"rendered":"[Chest\u53d1\u8868\u8bba\u6587]\uff1a\u4e13\u5bb6\u4e0e\u5927\u8bed\u8a00\u6a21\u578b\u751f\u6210\u7684\u673a\u68b0\u901a\u6c14\u9886\u57df\u591a\u9009\u9898\u7684\u8d28\u91cf"},"content":{"rendered":"\n

EDUCATION AND CLINICAL PRACTICE: ORIGINAL RESEARCH<\/strong><\/p>\n\n\n\n

Quality of Human Expert vs Large Language Model-Generated Multiple-Choice Questions in the Field of Mechanical Ventilation<\/h1>\n\n\n\n

Sami\u00a0Safadi,\u00a0Roxana\u00a0Amirahmadi,\u00a0Burton W.\u00a0Lee,\u00a0et al<\/h3>\n\n\n\n

Chest 2025; 168: 1425-1432<\/h3>\n\n\n\n

Abstract<\/h2>\n\n\n\n

Background<\/h3>\n\n\n\n

Although mechanical ventilation (MV) is a critical competency in critical care training, standardized methods for assessing MV-related knowledge are lacking. Traditional multiple-choice question (MCQ) development is resource intensive, and prior studies have suggested that generative AI tools could streamline question creation. However, the quality of AI-generated MCQs remains unclear.<\/p>\n\n\n\n

Research Question<\/h3>\n\n\n\n

Are MCQs generated by ChatGPT noninferior to human expert (HE)-created questions in terms of quality and relevance for MV education?<\/p>\n\n\n\n

Study Design and Methods<\/h3>\n\n\n\n

Three key MV topics were selected: Equation of Motion and Ohm\u2019s Law, Tau and Auto-PEEP, and Oxygenation. Fifteen learning objectives were used to generate 15 AI-written MCQs via a standardized prompt with ChatGPT-o1 (preview model; made available September 12, 2024). A group of 31 faculty experts, all of whom regularly teach MV, evaluated both AI- and HE-generated MCQs. Each MCQ was assessed based on its alignment with learning objectives, accuracy of chosen answer, clarity of the question stem, plausibility of distractor options, and difficulty level. The faculty members were blinded to the provenance of the MCQ questions. The noninferiority margin was predefined as 15% of the total possible score (\u20133.45).<\/p>\n\n\n\n

Results<\/h3>\n\n\n\n

AI-generated MCQs were statistically noninferior to the HE-written MCQs (95% upper CI, [\u20131.15, \u221e]). In additions, respondents were unable to reliably differentiate AI-generated MCQs from HE-written MCQs (P<\/em>\u00a0= .32).<\/p>\n\n\n\n

Table 1. Topics and Learning Objectives<\/p>\n\n\n\n

Topic<\/th>Learning Objective<\/th><\/tr><\/thead>
Equation of Motion and Ohm\u2019s Law<\/td>\u2022Use a clinical vignette to calculate static compliance\u2022Use a clinical vignette to apply the Equation of Motion to a patient with high airway pressures\u2022Use a clinical vignette to identify changes in lung static compliance and alveolar pressure\u2022Use a clinical vignette to demonstrate how the Equation of Motion explains the relationships among alveolar pressure, peak pressure, and respiratory muscular pressure (Pmusc)\u2022Understand the components of the Equation of Motion in a square wave volume control setting<\/td><\/tr>
Tau and Auto-PEEP<\/td>\u2022Understand the factors that determine how much volume remains in the lung at the end of expiration\u2022Use a clinical vignette to identify the relationship between compliance, resistance, and risk of auto-PEEP\u2022Recognize the most effective change a clinician can make to acutely treat auto-PEEP\u2022Understand that the expiratory time constant represents the time needed to exhale until 37% of tidal volume remains in the lungs\u2022Understand that a patient needs at least 3 expiratory time constants to fully exhale tidal volume<\/td><\/tr>
Oxygenation<\/td>\u2022Understand how to calculate the stress index in a patient who is mechanically ventilated\u2022Understand oxygen toxicity in a patient who is mechanically ventilated\u2022Understand the impact of higher PEEP on oxygenation and survival in patients with ARDS\u2022Understand the appropriate application of stress index monitoring for adjusting PEEP in a patient with severe ARDS. Present a clinical vignette\u2022Identify a clinical strategy to improve oxygenation and reduce mortality in ARDS<\/td><\/tr><\/tbody><\/table>
PEEP = positive end-expiratory pressure.<\/figcaption><\/figure>\n\n\n\n
\"\"\/<\/figure>\n\n\n\n

Table 2. Raters\u2019 Ability to Identify Source of Multiple-Choice Question<\/p>\n\n\n\n

Group<\/th>Guess: Artificial Intelligence<\/th>Guess: Human Expert<\/th>Total<\/th><\/tr><\/thead>
Group: Artificial Intelligence<\/th>256 (55.1%)<\/td>209 (45.0%)<\/td>465<\/td><\/tr>
Group: Human Expert<\/th>241 (51.8%)<\/td>224 (48.2%)<\/td>465<\/td><\/tr>
Total<\/th>497<\/td>433<\/td>930<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n

Interpretation<\/h3>\n\n\n\n

Our results suggest that AI-generated MCQs using ChatGPT-o1 are comparable in quality to those written by HEs. Given the time and resource-intensive nature of human MCQ development, AI-assisted question generation may serve as an efficient and scalable alternative for medical education assessment, even in highly specialized domains such as MV.<\/p>\n","protected":false},"excerpt":{"rendered":"

EDUCATION AND CLINICAL PRACTICE: ORIGINAL RESEARCH Qual […]<\/p>\n","protected":false},"author":3,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[32,23],"tags":[],"_links":{"self":[{"href":"https:\/\/csccm.org.cn\/index.php?rest_route=\/wp\/v2\/posts\/29957"}],"collection":[{"href":"https:\/\/csccm.org.cn\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/csccm.org.cn\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/csccm.org.cn\/index.php?rest_route=\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/csccm.org.cn\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=29957"}],"version-history":[{"count":1,"href":"https:\/\/csccm.org.cn\/index.php?rest_route=\/wp\/v2\/posts\/29957\/revisions"}],"predecessor-version":[{"id":29959,"href":"https:\/\/csccm.org.cn\/index.php?rest_route=\/wp\/v2\/posts\/29957\/revisions\/29959"}],"wp:attachment":[{"href":"https:\/\/csccm.org.cn\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=29957"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/csccm.org.cn\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=29957"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/csccm.org.cn\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=29957"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}