Original Investigation
March 31, 2025
Inclusion of Retracted Studies in Systematic Reviews and Meta-Analyses of Interventions: A Systematic Review and Meta-Analysis
Carolina Graña Possamai, Guillaume Cabanac, Elodie Perrodeau, et al
JAMA Intern Med. Published online March 31, 2025. doi:10.1001/jamainternmed.2025.0256
Question Do retracted studies impact results of systematic reviews with meta-analyses?
Findings This systematic review and meta-analysis identified 50 systematic reviews with 173 meta-analyses, each including at least 1 retracted study, published in 25 high–impact factor journals and calculated the effect estimates evolution rate for 163 meta-analyses (64 assessing the systematic review primary outcomes). For primary outcomes, effect estimates changed by at least 10% in 42% of meta-analyses, 30% in 25%, and 50% in 19%; overall (n = 163), effect estimates changed by at least 10% in 35%, 30% in 19%, and 50% in 14%.
Meaning Retracted studies can impact the results of meta-analyses.
Abstract
Importance Retractions are rising in the scientific literature, increasing the risk of reusing unreliable results.
Objectives To identify reports of systematic reviews that included retracted studies in their meta-analyses, and to assess the impact of these retracted studies on the results.
Design, Setting, and Participants In this systematic review and meta-analysis the Feet of Clay Detector tool was searched to identify all systematic reviews that reported at least 1 meta-analysis including at least 1 retracted study and were published in the 25 highest impact factor journals in medicine, general and internal, from January 2013 to April 2024. All effect estimates where the retracted study contributed to the analysis were identified. For each meta-analysis, the summary effect, including all studies and excluding retracted studies was calculated. The search was conducted on April 8, 2024.
Results Overall, 61 systematic reviews were identified that included retracted studies in their meta-analyses. Of these, 11 (18%) have been republished, retracted, or withdrawn. Data were extracted from 50 systematic reviews that included a total of 62 retracted studies. Retraction occurred after the publication in 37 systematic reviews (74%). Overall, 173 meta-analyses including the retracted study were identified; 70 of them (40%) were primary outcomes. One-hundred sixty-six meta-analyses were recalculated. Overall, 160 (96%) of the 166 recalculated effect estimates were within the CIs of the original effect. After exclusion of the retracted study, the statistical significance of the results changed in 18 meta-analyses (11%). The rate of evolution between effect estimates with and without retracted studies for 163 meta-analyses overall were calculated, including 64 addressing the primary outcomes of the systematic reviews. For primary outcomes (n = 64), the recalculated effect estimates changed by at least 10% in 27 meta-analyses (42%), 30% in 16 (25%), and 50% in 12 (19%). Overall (n = 163), effect estimates changed by at least 10% in 57 meta-analyses (35%), 30% in 31 (19%), and 50% in 23 (14%).
Conclusions and Relevance This study found that retracted studies have been included in systematic reviews and meta-analyses, with retractions occurring mainly after the publication of the systematic review. The inclusion of retracted studies can impact the results and interpretation of reviews. Quality control measures should be implemented to prevent the dissemination of unreliable data in scientific literature.
