Editorial
March 31, 2025
Mitigating the Impact of Retracted Studies in the Medical Literature
Cary P. Gross, Annette Flanagin, Eli N. Perencevich, et al
JAMA Intern Med. Published online March 31, 2025. doi:10.1001/jamainternmed.2025.0251
Medical publishing, and the scientific research ecosystem in general, face substantial challenges. Some are external, ranging from a relative decrease in research funding, to increased misinformation, politicization, and distrust of science and medicine. Yet some of the greatest problems facing science and science communication are coming from inside the house: financial and ideologic conflicts of interest, concerns about replicability of research and research integrity, the rise of predatory journals and papermills, and more recently, questions about the appropriate use of large language models.1-4
To be sure, the underlying pressure points of competing priorities, scarce resources, and evolving technologies are long-standing. Yet these external pressures on science have become acute and pronounced. Accordingly, the scientific establishment needs to ensure that its own house is in order to bolster the credibility of science and maintain the public’s trust. One critical domain is safeguarding the integrity of published peer-reviewed research. Retractions of scientific articles are often due to major pervasive errors or scientific misconduct (eg, plagiarism, fabrication, or falsification).5 However, authors and researchers who fail to identify or acknowledge retractions can unknowingly or neglectfully contribute to the unintentional spread of retracted research.6 Unfortunately, retractions of original articles are increasingly common, with a recent estimate of 10 000 articles retracted in 2023, more than double the number from 2021.7 There are now over 50 000 retractions in the Retraction Watch Database.8 While some of the increase may be related to increases in powerful search engines, along with use of artificial intelligence and heightened awareness and monitoring of articles by readers and integrity sleuths, concern exists about erosion of ethical standards in science and medicine.9
Retracted studies can have an impact long after they are retracted.10 In one analysis of over 9500 studies that cited a retracted study, the retracted study was acknowledged in fewer than 5% of occasions.11 In another study, authors contacted authors of 88 publications to alert them that their publication included a study that had been retracted.12They received a reply from half of the authors, yet only 9 publications had taken action (such as publishing notices or letters) within 1 year.
In this context, the study by Grana et al13 provides insights. The authors address the key question of how retractions could impact the body of evidence. They identified 61 systematic reviews with meta-analyses that had included at least 1 retracted study in a meta-analysis. Of these, only 11 were subsequently republished, retracted, or withdrawn. After scrutinizing the remaining 50 reviews, which included 173 meta-analyses, Grana et al13 repeated the meta-analyses after excluding the retracted studies. They found that the recalculated effect size was within the confidence interval of the original meta-analysis (that had included the retracted study) in 96% of the systematic reviews but that the statistical significance of the results changed in 18 meta-analyses (11%). In assessment of primary outcomes in 64 of the meta-analyses, they also found that inclusion of the retracted studies changed effect estimates by at least 10% in 27 meta-analyses (42%), 30% in 16 (25%), and 50% in 12 (19%). Hence, the retracted studies may have had less influence on the question of whether the intervention worked but instead on the effect size (ie, how well it worked).
What can we do to minimize the impact of retractions? It is helpful to consider the timing of the systematic review publication relative to the retraction date of the involved trials. Of the 50 reviews in the primary analysis, it is notable that the trial retractions had occurred before the publication of 13 of the systematic reviews. In this scenario, the key question is how the search for trials was conducted. Given that the retracted status of trials is not always clear in bibliographic databases, it is important that the authors of the systematic reviews use complementary approaches to ascertaining whether studies were retracted.14,15 It would be helpful to add specific retraction identification strategies to existing systematic review guidelines (eg, PRISMA checklists, Cochrane).15,16 The recent recommendation of the Communication of Retractions, Removals, and Expressions of Concern (CREC) Working Group of the National Information Standards Organization to prepend the titles of all retracted articles with the word “RETRACTED” may also help authors identify studies that should not be included in systematic reviews.17
Conversely, the majority of studies in the sample included by Grana et al13 were retracted after the systematic review was published. The authors identified 37 such reviews, with a median of 31 months between the review’s publication and the subsequent retraction of the primary study. In these cases, both the authors of the review and the journals have important roles. It is important for readers to know that a systematic review includes a study that has subsequently been retracted. In this instance, 3 of the systematic reviews in the study by Grana et al13 had been published by 2 JAMA Network journals.18-20 In each instance, the reanalysis excluding the retracted articles did not affect the results of the originally published systematic review. The following correction has been added to each of these 3 articles, with the authors’ approval: “This meta-analysis was published before 1 of the included studies [reference] was retracted. A reanalysis without that study does not affect the reported results.” It is not always feasible for authors to systematically repeat their analyses after excluding a retracted study, and responses will vary from case to case. However, journals should consider adding corrections (when reanalyses are feasible) or notifying readers that a retracted study was included in the originally published systematic review.
Science is essential, iterative, and, despite the growth of technology, an inherently human endeavor. Errors happen; retractions happen. It is critical that the medical publishing community not only strives to minimize issues related to scientific integrity but also continues to assess the broader impact of retractions beyond the retracted article and to provide open and transparent communication when retractions affect systematic reviews and meta-analyses. Indeed, a hallmark of rigorous science is that it evolves and self-corrects over time.
