Dian Zeng, Yiming Qin, Bin Sheng, et al<\/h3>\n\n\n\nJAMA. <\/em>Published online April 28, 2025. doi:10.1001\/jama.2025.6571<\/h3>\n\n\n\nThe large language models (LLMs) DeepSeek-V3 and DeepSeek-R1, developed by a subsidiary of a Chinese quantitative investment company, have the distinct advantages of being low cost and open source (adhering to the MIT Open Source License, which permits free commercial utilization and secondary development while mandating retention of the original copyright notice). This has substantially diminished the accessibility barriers for LLMs by enabling near-zero-cost start-up ventures, intellectual property protection, and transparent, collaborative innovation.<\/a><\/p>\n\n\n\nBy March 2025, DeepSeek application had surpassed 110 million downloads globally1<\/a><\/sup> and had been met with significant global interest.2<\/a><\/sup> Moreover, since its official launch on January 20, 2025, the DeepSeek-R1 model has significantly enhanced the performance of LLMs in logical reasoning tasks, even with minimal human-annotated training data.3<\/a><\/sup>DeepSeek\u2019s open-source (more accurately, open-weight) framework allows end users to fine-tune derivative models, accelerating the democratization and deployment of artificial intelligence (AI) in health care.<\/a><\/p>\n\n\n\nIn China, the rate of DeepSeek testing, implementation, and adoption has been unprecedented. According to Baidu, China\u2019s predominant search engine, global search queries for DeepSeek have surpassed those for OpenAI\u2019s ChatGPT and Google Gemini by 10 times since late January 2025. In the health care domain, as of a search conducted on March 9, 2025, more than 300 hospitals in China have adopted private, local, on-site deployments of DeepSeek, with attempts to integrate the LLM into real-world clinical and hospital-related tasks, including clinical diagnostics and decision support, patient education, the conduct of scientific research, and hospital management systems (Figure<\/a>). Many hospitals perceive such private, on-site deployment as a secure method for protecting data integrity and patient confidentiality, thus being acceptable to integrate within hospital systems. However, this rapid, unregulated adoption has outpaced China\u2019s overall regulatory oversight and governance framework, and critical questions have arisen: Are hospitals properly evaluating clinical safety and efficacy before implementation? Are there standardized frameworks to assess whether these models deliver their promised benefits across diverse clinical scenarios?<\/a><\/a><\/p>\n\n\n\nFigure. \u00a0DeepSeek Searches and Hospital Implementation Rates in China, January-February 2025<\/p>\n\n\n\n![\"\"\/](\"https:\/\/cdn.jamanetwork.com\/ama\/content_public\/journal\/jama\/939623\/jpp250005f1_1747933081.61587.png?Expires=1753803561&Signature=bwPphveelTaA2pLvUVgpPrRMvNhjWchbEZbsAIupHT3QQO54TW6cZLEZ3Ay8VBkiyjfCj7OO5cq2jCVhUlqXCCHewkBTloUiRJ3OgqGoRvHRqjiY4laqDvl5tptagzP-vX7LBqMznhzi71MIBrEtxTjQMPwjxvhjTBPWYS2A2bSIGQnNOTV~ZrWDnx8aGHk4lfXR83xggsrOR~QhqBNhN-1PZ0s3Oxn3t6~cF0xAkkbBjY4a64oBxwc3hgOPiie~KfCU7S615IYeu60C4dm-T3EC5SVJV2HQBjiCbVwc68HUApXQlSz6RiQaddC7H9hkVEBbzeB3CLcv6Vsi4pf8lw__&Key-Pair-Id=APKAIE5G5CRDK6RD3PGA\")
<\/figure>\n\n\n\nCorrelation between public interest and institutional adoption of DeepSeek in China's health care sector. Data combine search query analytics from the Baidu Index, a web service that quantifies search volume as a proxy for public interest, with independently collected data on hospital implementation rates. Visualization reveals 2 parallel growth curves: (1) the exponential increase in public awareness and interest in DeepSeek following its January 2025 release, and (2) the corresponding rapid adoption across Chinese hospital systems, which led to 80 hospitals implementing private local deployments within the first month (as of February 24, 2025). This relationship suggests potential reciprocal reinforcement between public discourse and institutional implementation decisions occurring in an environment of limited regulatory oversight.<\/p>\n\n\n\n
In this Perspective, we discuss key issues with DeepSeek\u2019s rapid adoption in China\u2019s hospitals and the critical concerns this rapid adoption raises. We advocate for evaluation frameworks and collaborative governance among AI developers, physicians, health care professionals, and policymakers to ensure ethical and secure deployment of LLMs in health care settings, not only in China but globally.<\/a><\/p>\n\n\n\nTrends in DeepSeek\u2019s Accelerating Adoption<\/p>\n\n\n\n
Chinese health care regulations explicitly mandate that physicians engage in comprehensive communication with patients or their families during diagnosis and treatment processes, while strictly prohibiting the implementation of AI-generated prescriptions.4<\/a><\/sup> It also imposes constraints on the deployment of AI in autonomous decision-making applications within core medical domains, such as prescription issuance. The historical constraints on data interoperability between Chinese hospital information systems and foreign LLMs, such as ChatGPT, can be attributed to the dual barriers of policy regulation and technological complexity.5<\/a><\/sup> Meanwhile, domestically developed LLMs have long been confined to academic research settings because of limitations in open-source ecosystems and suboptimal accuracy. These technical limitations, combined with high deployment costs, restricted adoption to a niche segment of top-tier hospitals in major cities. However, the launch of DeepSeek has generated an unprecedented wave of rapid adoption, and DeepSeek is increasingly integrated with existing hospital information systems without corresponding regulatory adaptation, potentially exposing a substantial volume of sensitive health data to inadequate safeguards within its operational workflows.<\/a><\/p>\n\n\n\nThe rapid adoption of DeepSeek is further amplified through social media discourse, creating pressure on health care institutions to implement these technologies to avoid appearing technologically backward. For example, on Weibo, China\u2019s largest social media platform, DeepSeek-related health care topics consistently achieve trending status, with headlines such as \u201cDoctors claim DeepSeek has the level of a top-tier hospital\u201d and \u201cDoctors claim that DeepSeek makes medical consultations more efficient.\u201d This public and isolated endorsement is manifesting in clinical encounters, with physicians increasingly reporting patients who present DeepSeek-generated treatment recommendations and insist on adherence to these AI-formulated care plans,6<\/a><\/sup> despite persistent concerns regarding clinical safety (eg, hallucinations), inadequate evaluation, and other unresolved implementation challenges.<\/a><\/p>\n\n\n\nClinical Safety and Evaluation: The Primary Challenges of DeepSeek Deployment<\/p>\n\n\n\n
Clinical safety and efficacy evaluation represent the foremost concerns in DeepSeek implementation. Despite its advanced reasoning capabilities facilitated through inference chains, DeepSeek has drawn significant attention for its propensity to generate hallucinations\u2014plausible but factually incorrect outputs.7<\/a><\/sup> Although no definitive causal relationship exists between reasoning capabilities and hallucination frequency, empirical evidence suggests that improvements in reasoning are often accompanied by increased hallucinatory instances.8<\/a><\/sup> This creates substantial clinical risk through multiple pathways.<\/a><\/p>\n\n\n\nWhen patients access AI-generated medical information without proper clinical context or guidance, they may use these outputs to evaluate the appropriateness of treatments recommended by their physicians. This can engender unwarranted skepticism and potentially introduce new sources of patient-physician conflict. For health care professionals, the risks manifest in 2 distinct ways: some clinicians may develop overreliance on DeepSeek, leading to uncritical adoption of its outputs and subsequent diagnostic errors or treatment biases.9<\/a><\/sup> Conversely, more cautious practitioners face increased cognitive burden when methodically verifying AI-generated conclusions against established clinical evidence, creating a paradoxical tension between efficiency and accuracy in time-sensitive clinical environments.10<\/a><\/sup> Moreover, there are dangers associated with downstream uses, which embed DeepSeek into specific products due to expanded attack surfaces, inadequate data handling by downstream developers, and uncontrolled model adaptation.<\/a><\/p>\n\n\n\nThe clinical safety implications of DeepSeek deployment in China are magnified by unique health care and technological factors. China\u2019s health care landscape is characterized by severe regional disparities in primary care infrastructure and an aging population of individuals older than 60 years exceeding 20%,11<\/a><\/sup> while simultaneously being one of the world\u2019s most digitally integrated societies, with more than 800 million smartphone users in an adult population of 1.16 billion and a 92% digital penetration rate in major cities.12<\/a><\/sup> This combination creates a perfect storm for clinical safety concerns: underserved populations with complex medical needs now have unprecedented access to AI-driven health recommendations, but often lack the clinical oversight needed for safe implementation. Patients increasingly perceive AI tools like DeepSeek as accessible, cost-effective alternatives to traditional medical consultation, potentially bypassing important clinical safeguards. Although applications utilizing DeepSeek typically include disclaimers that recommendations are for reference only, these warnings may be insufficient given the varying health literacy levels across China\u2019s population and the limited regulatory oversight of AI-generated health recommendations, particularly for over-the-counter medications. This context underscores why standardized clinical safety evaluation frameworks are not merely academic concerns but urgent necessities in China\u2019s rapidly evolving health care AI landscape.<\/a><\/p>\n\n\n\nA 2025 systematic review by Huo et al13<\/a><\/sup> found that less than one-third of studies have addressed the ethical (32.8%) and patient safety (32.1%) implications of clinical LLM integration, while only 16.1% have addressed the regulatory gaps surrounding LLMs in health care. Moreover, as highlighted in a comprehensive JAMA<\/em> Review of 519 publications on LLMs in health care, evaluations are frequently inadequate\u2014they often fail to use real patient data, do not quantify biases, neglect to cover a broad range of clinical scenarios, and rarely report standardized performance metrics.14<\/a><\/sup><\/a><\/p>\n\n\n\nBeyond clinical accuracy considerations, the quality of patient-AI interaction during consultations represents another critical dimension of LLM implementation in health care. Research conducted by Tu et al in 2025 showed that LLM-based diagnostic AI systems are capable of matching or even surpassing human clinicians in terms of communication quality and empathetic engagement during text-based medical consultations.15<\/a><\/sup> Although DeepSeek\u2019s rapid adoption may leverage these potential benefits, the evaluation of these interpersonal capabilities across diverse patient populations and clinical contexts remains largely unaddressed in current implementation practices. As with diagnostic accuracy, standardized frameworks for assessing AI communication effectiveness are also essential prior to widespread clinical integration.<\/a><\/p>\n\n\n\nAlthough clinical safety remains the primary concern, DeepSeek\u2019s rapid deployment has also intensified existing challenges around data security and patient privacy. Although private local deployments are often assumed to inherently mitigate these risks compared with cloud-based models, this approach actually shifts security responsibilities to individual health care facilities, many of which lack comprehensive cybersecurity infrastructure. DeepSeek\u2019s implementation with existing hospital information systems, when conducted without proper safeguards, may create new vulnerabilities for sensitive patient data, particularly given the model\u2019s extensive parameter requirements and the corresponding hardware configurations needed for effective deployment.<\/a><\/p>\n\n\n\nWhen considering these evaluation challenges, the implementation of DeepSeek or any LLM in health care can be distilled into 2 essential components: first, clearly defining the specific clinical tasks for which AI assistance provides meaningful value, and second, rigorously evaluating whether these models reliably accomplish those tasks across diverse clinical scenarios. Recent efforts, such as the MedHELM16<\/a> <\/sup>initiative, have made significant progress by defining approximately 120 health care tasks with corresponding evaluation scenarios and standard datasets, establishing crucial benchmarks for standardized LLM performance assessment in clinical contexts. Building on these insights, we propose organizing health care AI applications into 3 primary domains requiring distinct evaluation approaches: clinical decision support, patient management and service, and research and teaching (Table<\/a>).<\/a><\/p>\n\n\n\nTable. Clinical Safety, Data Security, and Patient Privacy Risks and Solutions When Deploying Artificial Intelligence (AI)\u2013Driven Health Care Systems<\/p>\n\n\n\n![\"\"\/](\"https:\/\/cdn.jamanetwork.com\/ama\/content_public\/journal\/jama\/939623\/jpp250005t1_1747933081.66195.png?Expires=1753803561&Signature=K4fk77Selq2ndHeSybkDdTcyY9A3mgEmuVNY-HjgdyQUpAGmClkcAY7izlEhBbYvFFPevmg-WQywsNDwqqNE6DQyof2k2yug3akv0KtojlkguBT2CO-qcMAEN5sB9pbCZduW3k1mbWuBDtlxZQzjcyoS62NINS9~OpySG4y0Z2F3gZ82Gx-hosIt5MajICtpNzwt3uSU26WRWmGtxtyM9OmKc6fZSuFdltcceuBkQbSLIp~~Yx~OHM9TZFnkaQZ6f~q4ExmIkUpnlxgNTMPS8-B7ujBLHDnluXFvIeuwf5oRHCrb~Y8xOh4mzMdquk7dOS8vqPft09vIWCmndxEleg__&Key-Pair-Id=APKAIE5G5CRDK6RD3PGA\")
<\/figure>\n\n\n\nThe Need for a Framework for Trustworthy AI Implementation in Health Care Settings in China<\/p>\n\n\n\n
The health care challenges associated with DeepSeek are inextricably linked to China\u2019s dual health system paradox: the imperative to leverage AI to bridge resource gaps vs the necessity to safeguard patient welfare in an environment where technology adoption outpaces regulatory maturation. The rapid integration of DeepSeek into China\u2019s health care systems has proceeded at a pace that appears too soon and too fast, necessitating an urgent need to proactively develop strategies to address clinical safety, data security, and patient privacy challenges while maximizing the transformative potential of AI and LLMs in health care.17<\/a><\/sup><\/a><\/p>\n\n\n\nBased on our analysis of DeepSeek implementations in Chinese hospitals and drawing upon established guidelines from the European TRAIN Initiative18<\/a><\/sup>; FUTURE-AI Consortium19<\/a><\/sup>; Blueprint for Trustworthy AI from the Coalition for Health AI20<\/a><\/sup>; US Department of Health and Human Services Strategic Plan for the Use of Artificial Intelligence in Health, Human Services, and Public Health21<\/a><\/sup>; MedHELM16<\/a><\/sup>; and OECD Steering AI\u2019s future,22<\/a><\/sup> we advocate for a structured approach to health care AI evaluation and implementation centered on the 3 clinical domains outlined above: clinical decision support, patient management and service, and research and teaching. Each domain requires distinct evaluation methodologies tailored to its specific clinical contexts and potential safety challenges, creating a comprehensive framework that addresses the multifaceted nature of health care AI applications.<\/a><\/p>\n\n\n\nEstablishing reliability boundaries for AI applications in medical diagnosis is essential to address clinical safety concerns across these domains. Health care institutions implementing DeepSeek or similar LLMs should adopt several evidence-based implementation strategies. First, institutions should establish task-specific evaluation protocols by defining clear success metrics for each clinical application, aligning with established clinical standards; the Duke University ABCDS framework23<\/a><\/sup> exemplifies this approach by evaluating AI tools based on clinically relevant outcomes rather than technical capabilities alone. Second, implementation should include multistage verification processes with rigorous preimplementation testing using diverse clinical scenarios reflecting population heterogeneity, followed by ongoing monitoring during clinical use to detect both preimplementation limitations and emergent performance issues. Third, health care institutions should develop integrated education approaches that combine patient education about AI limitations through targeted materials during care episodes with institution-wide training programs for clinicians on effective AI utilization,24<\/a><\/sup> ensuring AI tools supplement rather than replace clinical judgment. Fourth, robust governance structures must be established for local DeepSeek deployments, including comprehensive data governance protocols that extend beyond traditional health care data protections with defined access controls, secure development environments, and regular vulnerability assessments specific to LLM implementations.25<\/a><\/sup> Fifth, health care institutions should participate in collaborative benchmarking initiatives that develop shared evaluation frameworks and benchmarks, addressing challenges that transcend individual settings and accelerating collective learning about best practices for safe AI implementation.<\/a><\/p>\n\n\n\nThe implementation of DeepSeek in Chinese health care settings epitomizes a critical paradigm shift where AI-driven medical solutions transcend the traditional triadic relationship among patients, clinicians, and algorithms. This transformation necessitates a nuanced examination of 2 interrelated concerns: whether public perception may become skewed due to sensationalized media narratives, and whether such misalignment could precipitate malicious exploitation or excessive utilization of AI services by stakeholders. This comprehensive framework underscores the imperative for a tripartite strategy: rigorous predeployment evaluation, robust postmarket surveillance, and balanced approaches to maximizing benefits while minimizing risks across clinical, managerial, and ethical domains.<\/a><\/p>\n\n\n\nConclusions<\/p>\n\n\n\n
DeepSeek is seen as a low-cost and open-source AI model and has generated excitement globally but particularly so in China. DeepSeek\u2019s rapid adoption in China\u2019s hospital systems presents a common paradox in health care technology: balancing the opportunity to rapidly integrate innovative technology to transform health care with the potential risks and dangers that accompany such rapid adoption (\u201cfirst, do no harm\u201d). We propose global collaborative effort among AI developers, physicians, health care administrators, and policymakers to ensure that new AI technology, particularly frontier LLMs that are now inexpensive and widely available, can be used as a trustworthy and safe tool in health care, rather than a potential source of harm. Only through a balanced approach incorporating robust clinical safety with regulatory oversight can new AI models truly realize their potential to transform health care.<\/p>\n","protected":false},"excerpt":{"rendered":"
Perspective AI in Medicine April 28, 20 […]<\/p>\n","protected":false},"author":3,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[24,23],"tags":[],"_links":{"self":[{"href":"https:\/\/csccm.org.cn\/index.php?rest_route=\/wp\/v2\/posts\/28326"}],"collection":[{"href":"https:\/\/csccm.org.cn\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/csccm.org.cn\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/csccm.org.cn\/index.php?rest_route=\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/csccm.org.cn\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=28326"}],"version-history":[{"count":2,"href":"https:\/\/csccm.org.cn\/index.php?rest_route=\/wp\/v2\/posts\/28326\/revisions"}],"predecessor-version":[{"id":28686,"href":"https:\/\/csccm.org.cn\/index.php?rest_route=\/wp\/v2\/posts\/28326\/revisions\/28686"}],"wp:attachment":[{"href":"https:\/\/csccm.org.cn\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=28326"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/csccm.org.cn\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=28326"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/csccm.org.cn\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=28326"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
The large language models (LLMs) DeepSeek-V3 and DeepSeek-R1, developed by a subsidiary of a Chinese quantitative investment company, have the distinct advantages of being low cost and open source (adhering to the MIT Open Source License, which permits free commercial utilization and secondary development while mandating retention of the original copyright notice). This has substantially diminished the accessibility barriers for LLMs by enabling near-zero-cost start-up ventures, intellectual property protection, and transparent, collaborative innovation.<\/a><\/p>\n\n\n\n By March 2025, DeepSeek application had surpassed 110 million downloads globally1<\/a><\/sup> and had been met with significant global interest.2<\/a><\/sup> Moreover, since its official launch on January 20, 2025, the DeepSeek-R1 model has significantly enhanced the performance of LLMs in logical reasoning tasks, even with minimal human-annotated training data.3<\/a><\/sup>DeepSeek\u2019s open-source (more accurately, open-weight) framework allows end users to fine-tune derivative models, accelerating the democratization and deployment of artificial intelligence (AI) in health care.<\/a><\/p>\n\n\n\n In China, the rate of DeepSeek testing, implementation, and adoption has been unprecedented. According to Baidu, China\u2019s predominant search engine, global search queries for DeepSeek have surpassed those for OpenAI\u2019s ChatGPT and Google Gemini by 10 times since late January 2025. In the health care domain, as of a search conducted on March 9, 2025, more than 300 hospitals in China have adopted private, local, on-site deployments of DeepSeek, with attempts to integrate the LLM into real-world clinical and hospital-related tasks, including clinical diagnostics and decision support, patient education, the conduct of scientific research, and hospital management systems (Figure<\/a>). Many hospitals perceive such private, on-site deployment as a secure method for protecting data integrity and patient confidentiality, thus being acceptable to integrate within hospital systems. However, this rapid, unregulated adoption has outpaced China\u2019s overall regulatory oversight and governance framework, and critical questions have arisen: Are hospitals properly evaluating clinical safety and efficacy before implementation? Are there standardized frameworks to assess whether these models deliver their promised benefits across diverse clinical scenarios?<\/a><\/a><\/p>\n\n\n\n Figure. \u00a0DeepSeek Searches and Hospital Implementation Rates in China, January-February 2025<\/p>\n\n\n\n Correlation between public interest and institutional adoption of DeepSeek in China's health care sector. Data combine search query analytics from the Baidu Index, a web service that quantifies search volume as a proxy for public interest, with independently collected data on hospital implementation rates. Visualization reveals 2 parallel growth curves: (1) the exponential increase in public awareness and interest in DeepSeek following its January 2025 release, and (2) the corresponding rapid adoption across Chinese hospital systems, which led to 80 hospitals implementing private local deployments within the first month (as of February 24, 2025). This relationship suggests potential reciprocal reinforcement between public discourse and institutional implementation decisions occurring in an environment of limited regulatory oversight.<\/p>\n\n\n\n In this Perspective, we discuss key issues with DeepSeek\u2019s rapid adoption in China\u2019s hospitals and the critical concerns this rapid adoption raises. We advocate for evaluation frameworks and collaborative governance among AI developers, physicians, health care professionals, and policymakers to ensure ethical and secure deployment of LLMs in health care settings, not only in China but globally.<\/a><\/p>\n\n\n\n Trends in DeepSeek\u2019s Accelerating Adoption<\/p>\n\n\n\n Chinese health care regulations explicitly mandate that physicians engage in comprehensive communication with patients or their families during diagnosis and treatment processes, while strictly prohibiting the implementation of AI-generated prescriptions.4<\/a><\/sup> It also imposes constraints on the deployment of AI in autonomous decision-making applications within core medical domains, such as prescription issuance. The historical constraints on data interoperability between Chinese hospital information systems and foreign LLMs, such as ChatGPT, can be attributed to the dual barriers of policy regulation and technological complexity.5<\/a><\/sup> Meanwhile, domestically developed LLMs have long been confined to academic research settings because of limitations in open-source ecosystems and suboptimal accuracy. These technical limitations, combined with high deployment costs, restricted adoption to a niche segment of top-tier hospitals in major cities. However, the launch of DeepSeek has generated an unprecedented wave of rapid adoption, and DeepSeek is increasingly integrated with existing hospital information systems without corresponding regulatory adaptation, potentially exposing a substantial volume of sensitive health data to inadequate safeguards within its operational workflows.<\/a><\/p>\n\n\n\n The rapid adoption of DeepSeek is further amplified through social media discourse, creating pressure on health care institutions to implement these technologies to avoid appearing technologically backward. For example, on Weibo, China\u2019s largest social media platform, DeepSeek-related health care topics consistently achieve trending status, with headlines such as \u201cDoctors claim DeepSeek has the level of a top-tier hospital\u201d and \u201cDoctors claim that DeepSeek makes medical consultations more efficient.\u201d This public and isolated endorsement is manifesting in clinical encounters, with physicians increasingly reporting patients who present DeepSeek-generated treatment recommendations and insist on adherence to these AI-formulated care plans,6<\/a><\/sup> despite persistent concerns regarding clinical safety (eg, hallucinations), inadequate evaluation, and other unresolved implementation challenges.<\/a><\/p>\n\n\n\n Clinical Safety and Evaluation: The Primary Challenges of DeepSeek Deployment<\/p>\n\n\n\n Clinical safety and efficacy evaluation represent the foremost concerns in DeepSeek implementation. Despite its advanced reasoning capabilities facilitated through inference chains, DeepSeek has drawn significant attention for its propensity to generate hallucinations\u2014plausible but factually incorrect outputs.7<\/a><\/sup> Although no definitive causal relationship exists between reasoning capabilities and hallucination frequency, empirical evidence suggests that improvements in reasoning are often accompanied by increased hallucinatory instances.8<\/a><\/sup> This creates substantial clinical risk through multiple pathways.<\/a><\/p>\n\n\n\n When patients access AI-generated medical information without proper clinical context or guidance, they may use these outputs to evaluate the appropriateness of treatments recommended by their physicians. This can engender unwarranted skepticism and potentially introduce new sources of patient-physician conflict. For health care professionals, the risks manifest in 2 distinct ways: some clinicians may develop overreliance on DeepSeek, leading to uncritical adoption of its outputs and subsequent diagnostic errors or treatment biases.9<\/a><\/sup> Conversely, more cautious practitioners face increased cognitive burden when methodically verifying AI-generated conclusions against established clinical evidence, creating a paradoxical tension between efficiency and accuracy in time-sensitive clinical environments.10<\/a><\/sup> Moreover, there are dangers associated with downstream uses, which embed DeepSeek into specific products due to expanded attack surfaces, inadequate data handling by downstream developers, and uncontrolled model adaptation.<\/a><\/p>\n\n\n\n The clinical safety implications of DeepSeek deployment in China are magnified by unique health care and technological factors. China\u2019s health care landscape is characterized by severe regional disparities in primary care infrastructure and an aging population of individuals older than 60 years exceeding 20%,11<\/a><\/sup> while simultaneously being one of the world\u2019s most digitally integrated societies, with more than 800 million smartphone users in an adult population of 1.16 billion and a 92% digital penetration rate in major cities.12<\/a><\/sup> This combination creates a perfect storm for clinical safety concerns: underserved populations with complex medical needs now have unprecedented access to AI-driven health recommendations, but often lack the clinical oversight needed for safe implementation. Patients increasingly perceive AI tools like DeepSeek as accessible, cost-effective alternatives to traditional medical consultation, potentially bypassing important clinical safeguards. Although applications utilizing DeepSeek typically include disclaimers that recommendations are for reference only, these warnings may be insufficient given the varying health literacy levels across China\u2019s population and the limited regulatory oversight of AI-generated health recommendations, particularly for over-the-counter medications. This context underscores why standardized clinical safety evaluation frameworks are not merely academic concerns but urgent necessities in China\u2019s rapidly evolving health care AI landscape.<\/a><\/p>\n\n\n\n A 2025 systematic review by Huo et al13<\/a><\/sup> found that less than one-third of studies have addressed the ethical (32.8%) and patient safety (32.1%) implications of clinical LLM integration, while only 16.1% have addressed the regulatory gaps surrounding LLMs in health care. Moreover, as highlighted in a comprehensive JAMA<\/em> Review of 519 publications on LLMs in health care, evaluations are frequently inadequate\u2014they often fail to use real patient data, do not quantify biases, neglect to cover a broad range of clinical scenarios, and rarely report standardized performance metrics.14<\/a><\/sup><\/a><\/p>\n\n\n\n Beyond clinical accuracy considerations, the quality of patient-AI interaction during consultations represents another critical dimension of LLM implementation in health care. Research conducted by Tu et al in 2025 showed that LLM-based diagnostic AI systems are capable of matching or even surpassing human clinicians in terms of communication quality and empathetic engagement during text-based medical consultations.15<\/a><\/sup> Although DeepSeek\u2019s rapid adoption may leverage these potential benefits, the evaluation of these interpersonal capabilities across diverse patient populations and clinical contexts remains largely unaddressed in current implementation practices. As with diagnostic accuracy, standardized frameworks for assessing AI communication effectiveness are also essential prior to widespread clinical integration.<\/a><\/p>\n\n\n\n Although clinical safety remains the primary concern, DeepSeek\u2019s rapid deployment has also intensified existing challenges around data security and patient privacy. Although private local deployments are often assumed to inherently mitigate these risks compared with cloud-based models, this approach actually shifts security responsibilities to individual health care facilities, many of which lack comprehensive cybersecurity infrastructure. DeepSeek\u2019s implementation with existing hospital information systems, when conducted without proper safeguards, may create new vulnerabilities for sensitive patient data, particularly given the model\u2019s extensive parameter requirements and the corresponding hardware configurations needed for effective deployment.<\/a><\/p>\n\n\n\n When considering these evaluation challenges, the implementation of DeepSeek or any LLM in health care can be distilled into 2 essential components: first, clearly defining the specific clinical tasks for which AI assistance provides meaningful value, and second, rigorously evaluating whether these models reliably accomplish those tasks across diverse clinical scenarios. Recent efforts, such as the MedHELM16<\/a> <\/sup>initiative, have made significant progress by defining approximately 120 health care tasks with corresponding evaluation scenarios and standard datasets, establishing crucial benchmarks for standardized LLM performance assessment in clinical contexts. Building on these insights, we propose organizing health care AI applications into 3 primary domains requiring distinct evaluation approaches: clinical decision support, patient management and service, and research and teaching (Table<\/a>).<\/a><\/p>\n\n\n\n Table. Clinical Safety, Data Security, and Patient Privacy Risks and Solutions When Deploying Artificial Intelligence (AI)\u2013Driven Health Care Systems<\/p>\n\n\n\n The Need for a Framework for Trustworthy AI Implementation in Health Care Settings in China<\/p>\n\n\n\n The health care challenges associated with DeepSeek are inextricably linked to China\u2019s dual health system paradox: the imperative to leverage AI to bridge resource gaps vs the necessity to safeguard patient welfare in an environment where technology adoption outpaces regulatory maturation. The rapid integration of DeepSeek into China\u2019s health care systems has proceeded at a pace that appears too soon and too fast, necessitating an urgent need to proactively develop strategies to address clinical safety, data security, and patient privacy challenges while maximizing the transformative potential of AI and LLMs in health care.17<\/a><\/sup><\/a><\/p>\n\n\n\n Based on our analysis of DeepSeek implementations in Chinese hospitals and drawing upon established guidelines from the European TRAIN Initiative18<\/a><\/sup>; FUTURE-AI Consortium19<\/a><\/sup>; Blueprint for Trustworthy AI from the Coalition for Health AI20<\/a><\/sup>; US Department of Health and Human Services Strategic Plan for the Use of Artificial Intelligence in Health, Human Services, and Public Health21<\/a><\/sup>; MedHELM16<\/a><\/sup>; and OECD Steering AI\u2019s future,22<\/a><\/sup> we advocate for a structured approach to health care AI evaluation and implementation centered on the 3 clinical domains outlined above: clinical decision support, patient management and service, and research and teaching. Each domain requires distinct evaluation methodologies tailored to its specific clinical contexts and potential safety challenges, creating a comprehensive framework that addresses the multifaceted nature of health care AI applications.<\/a><\/p>\n\n\n\n Establishing reliability boundaries for AI applications in medical diagnosis is essential to address clinical safety concerns across these domains. Health care institutions implementing DeepSeek or similar LLMs should adopt several evidence-based implementation strategies. First, institutions should establish task-specific evaluation protocols by defining clear success metrics for each clinical application, aligning with established clinical standards; the Duke University ABCDS framework23<\/a><\/sup> exemplifies this approach by evaluating AI tools based on clinically relevant outcomes rather than technical capabilities alone. Second, implementation should include multistage verification processes with rigorous preimplementation testing using diverse clinical scenarios reflecting population heterogeneity, followed by ongoing monitoring during clinical use to detect both preimplementation limitations and emergent performance issues. Third, health care institutions should develop integrated education approaches that combine patient education about AI limitations through targeted materials during care episodes with institution-wide training programs for clinicians on effective AI utilization,24<\/a><\/sup> ensuring AI tools supplement rather than replace clinical judgment. Fourth, robust governance structures must be established for local DeepSeek deployments, including comprehensive data governance protocols that extend beyond traditional health care data protections with defined access controls, secure development environments, and regular vulnerability assessments specific to LLM implementations.25<\/a><\/sup> Fifth, health care institutions should participate in collaborative benchmarking initiatives that develop shared evaluation frameworks and benchmarks, addressing challenges that transcend individual settings and accelerating collective learning about best practices for safe AI implementation.<\/a><\/p>\n\n\n\n The implementation of DeepSeek in Chinese health care settings epitomizes a critical paradigm shift where AI-driven medical solutions transcend the traditional triadic relationship among patients, clinicians, and algorithms. This transformation necessitates a nuanced examination of 2 interrelated concerns: whether public perception may become skewed due to sensationalized media narratives, and whether such misalignment could precipitate malicious exploitation or excessive utilization of AI services by stakeholders. This comprehensive framework underscores the imperative for a tripartite strategy: rigorous predeployment evaluation, robust postmarket surveillance, and balanced approaches to maximizing benefits while minimizing risks across clinical, managerial, and ethical domains.<\/a><\/p>\n\n\n\n Conclusions<\/p>\n\n\n\n DeepSeek is seen as a low-cost and open-source AI model and has generated excitement globally but particularly so in China. DeepSeek\u2019s rapid adoption in China\u2019s hospital systems presents a common paradox in health care technology: balancing the opportunity to rapidly integrate innovative technology to transform health care with the potential risks and dangers that accompany such rapid adoption (\u201cfirst, do no harm\u201d). We propose global collaborative effort among AI developers, physicians, health care administrators, and policymakers to ensure that new AI technology, particularly frontier LLMs that are now inexpensive and widely available, can be used as a trustworthy and safe tool in health care, rather than a potential source of harm. Only through a balanced approach incorporating robust clinical safety with regulatory oversight can new AI models truly realize their potential to transform health care.<\/p>\n","protected":false},"excerpt":{"rendered":" Perspective AI in Medicine April 28, 20 […]<\/p>\n","protected":false},"author":3,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[24,23],"tags":[],"_links":{"self":[{"href":"https:\/\/csccm.org.cn\/index.php?rest_route=\/wp\/v2\/posts\/28326"}],"collection":[{"href":"https:\/\/csccm.org.cn\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/csccm.org.cn\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/csccm.org.cn\/index.php?rest_route=\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/csccm.org.cn\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=28326"}],"version-history":[{"count":2,"href":"https:\/\/csccm.org.cn\/index.php?rest_route=\/wp\/v2\/posts\/28326\/revisions"}],"predecessor-version":[{"id":28686,"href":"https:\/\/csccm.org.cn\/index.php?rest_route=\/wp\/v2\/posts\/28326\/revisions\/28686"}],"wp:attachment":[{"href":"https:\/\/csccm.org.cn\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=28326"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/csccm.org.cn\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=28326"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/csccm.org.cn\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=28326"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}<\/figure>\n\n\n\n<\/figure>\n\n\n\n