Research Letter
Statistics and Research Methods
Confounder Selection in Observational Studies in High-Impact Medical and Epidemiological Journals
Luis C. L. Correia, Rafael F. Mascarenhas, Felipe S. C. De Menezes, et al
JAMA Netw Open 2025;8;(7):e2524176. doi:10.1001/jamanetworkopen.2025.24176
Introduction
There is growing interest in using observational data and methods to evaluate questions about the causal effects of exposures on outcomes.1 However, observational studies rely on strong assumptions to support causal conclusions, including that of no uncontrolled confounding.1 Traditional statistical approaches, such as the change-in-estimate strategy, are now considered inadequate for confounder selection,2 and current recommendations support an informed approach based on theoretical models—described in text or illustrated with directed acyclic graphs (DAGs)—that consider the associations between exposures and outcomes.2,3 Following a 2004 study that found observational studies often lack sufficient rationale for the selection of confounders,4 the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) reporting guideline emphasized the need for studies to clearly define potential confounders.5 We evaluated whether there have been improvements over time in the methods reported for selecting confounders to control for in observational studies published in the highest impact factor medical and epidemiological journals.
Methods
We conducted a cross-sectional study and identified the 10 highest impact factor medical (n = 5) and epidemiological (n = 5) journals according to InCites Journal Citation Reports. For each journal, we reviewed all PubMed-indexed articles from 2003, 2013, and 2023 to identify observational studies evaluating exposure-outcome associations in which confounder adjustment would be expected, excluding those that were descriptive, predictive, and quasi-experimental (ie, that primarily address confounding through design-based approaches) (eFigure in Supplement 1). We randomly selected one-half of the articles in each journal and year for full-text evaluation and identified key study characteristics, including design, exposure type, and funding source. We classified the methods reported by each study to select confounders: no confounder adjustment, adjustment but confounders not specified, confounders selected without justification, confounders selected on the basis of an established association with the outcome, confounders selected on the basis of statistical criteria (eg, imbalance between exposure groups, change-in-estimate strategy, or stepwise regression), or confounders selected on the basis of a causal model, either depicted by a DAG or explained in the text. This study utilized publicly available data and did not require ethics approval or patient consent, in accordance with 45 CFR §46. We followed the STROBE reporting guideline for cross-sectional studies.5 Data were analyzed using R version 4.4.0 (R Project for Statistical Computing).
Results
We identified 623 eligible observational studies, including 197 (31.6%) published in medical and 426 (68.4%) published in epidemiological journals. Of these, 22 (3.5%) did not report adjusting for confounders, 18 (2.9%) did not specify which confounders were selected, 281 (45.1%) reported selection of confounders without justification, 139 (22.3%) reported selection of confounders on the basis of an established association with the outcome, 121 (19.4%) reported selection of confounders on the basis of statistical criteria, and 42 (6.7%) reported selection of confounders on the basis of a causal model (35 used DAGs and 7 provided an explanation in their text) (Table 1).
Table 1. Methods Used by Observational Studies to Select for Confounders
| Year and journal type | Total No. of studiesa | Studies, No. (%) | |||||
|---|---|---|---|---|---|---|---|
| No confounder adjustment | Adjustment with confounders not specified | Confounders selected without justification | Confounders selected on the basis of an established association with the outcome | Confounders selected on the basis of to statistical criteriab | Confounders selected on the basis of causal model | ||
| 2003 | |||||||
| Medical journal | 82 | 13 (15.8) | 7 (8.5) | 37 (45.1) | 14 (17.1) | 11 (13.4) | 0 |
| Epidemiological journal | 146 | 6 (4.1) | 0 | 74 (50.7) | 34 (23.3) | 32 (21.9) | 0 |
| Total | 228 | 19 (8.3) | 7 (3.1) | 111 (48.7) | 48 (21.1) | 43 (18.9) | 0 |
| 2013 | |||||||
| Medical journal | 70 | 0 | 8 (11.4) | 27 (38.6) | 14 (20.0) | 19 (27.1) | 2 (2.8) |
| Epidemiological journal | 161 | 3 (1.9) | 1 (0.6) | 75 (46.6) | 47 (29.2) | 32 (19.9) | 3 (1.9) |
| Total | 231 | 3 (1.3) | 9 (3.9) | 102 (44.2) | 61 (26.4) | 51 (22.1) | 5 (2.2) |
| 2023 | |||||||
| Medical journal | 45 | 0 | 2 (4.4) | 22 (48.9) | 3 (6.7) | 9 (20.0) | 9 (20.0) |
| Epidemiol journal | 119 | 0 | 0 | 46 (38.7) | 27 (22.7) | 18 (15.1) | 28 (23.5) |
| Total | 164 | 0 | 2 (1.2) | 68 (41.5) | 30 (18.2) | 27 (16.5) | 37 (22.6) |
| All years and journals | 623 | 22 (3.5) | 18 (2.9) | 281 (45.1) | 139 (22.3) | 121 (19.4) | 42 (6.7) |
The selection of confounders without justification remained relatively constant between 2003 and 2023—from 111 of 228 (48.7%) to 68 of 164 (41.5%)—while the use of a causal model to identify confounders increased—from 0 of 228 to 37 of 164 (22.6%). Differences in the methods used to select confounders were observed across journal type and study design (Table 2).
Table 2. Comparison of Study Characteristics by Confounder Selection Method
| Characteristic | Total No. of studies | Studies, No. (%) | |||||
|---|---|---|---|---|---|---|---|
| No confounder adjustment | Adjustment with confounders not specified | Confounders selected without justification | Confounders selected on the basis of an established association with the outcome | Confounders selected on the basis of statistical criteriaa | Confounders selected on the basis of a causal model | ||
| Type of journal | |||||||
| Medical | 197 | 13 (6.6) | 17 (8.6) | 86 (43.7) | 31 (15.7) | 39 (19.8) | 11 (5.6) |
| Epidemiological | 426 | 9 (2.1) | 1 (0.2) | 195 (45.8) | 108 (25.4) | 82 (19.3) | 31 (7.3) |
| Type of exposure | |||||||
| Devices | 9 | 0 | 1 (11.1) | 5 (55.6) | 2 (22.2) | 1 (11.1) | 0 |
| Drugs and biologics | 89 | 6 (6.7) | 2 (2.2) | 35 (39.3) | 17 (19.1) | 23 (25.8) | 6 (6.7) |
| Clinical risk factors | 191 | 11 (5.8) | 8 (4.2) | 91 (47.6) | 34 (17.8) | 34 (17.8) | 13 (6.8) |
| Behavioral exposures | 92 | 1 (1.1) | 1 (1.1) | 46 (50.0) | 28 (30.4) | 8 (8.7) | 8 (8.7) |
| Psychosocial and environmental exposures | 168 | 3 (1.8) | 2 (1.2) | 64 (38.1) | 46 (27.4) | 41 (24.4) | 12 (7.1) |
| Other | 74 | 1 (1.4) | 4 (5.4) | 40 (54.1) | 12 (16.2) | 14 (18.9) | 3 (4.1) |
| Study design | |||||||
| Cohort | 434 | 11 (2.5) | 11 (2.5) | 197 (45.4) | 97 (22.4) | 80 (18.4) | 38 (8.8) |
| Case-control | 94 | 5 (5.3) | 0 | 45 (47.9) | 19 (20.2) | 23 (24.5) | 2 (2.1) |
| Cross-sectional | 95 | 6 (6.3) | 7 (7.4) | 39 (41.1) | 23 (24.2) | 18 (18.95) | 2 (2.1) |
| Funding source | |||||||
| Industry | 22 | 1 (4.6) | 2 (9.1) | 11 (50.0) | 5 (22.7) | 3 (13.6) | 0 |
| Nonindustry | 504 | 18 (3.6) | 14 (2.8) | 212 (42.1) | 119 (23.6) | 101 (20.0) | 40 (7.9) |
| No funding | 59 | 0 | 1 (1.7) | 37 (62.7) | 8 (13.6) | 11 (18.6) | 2 (3.4) |
| Not mentioned | 38 | 3 (7.9) | 1 (2.6) | 21 (55.3) | 7 (18.4) | 6 (15.8) | 0 |
Discussion
This cross-sectional study found that among 623 observational studies published over the past 2 decades in the highest impact factor medical and epidemiological journals, approximately one-half selected confounders without reporting justification. Although reporting of causal models, such as DAGs, has increased over time, fewer than one-quarter did so in 2023, raising concerns about how confounders are selected and justified in observational studies.
Study limitations include a focus on reported rather than actual methods, and limited generalizability across journals and fields, as reporting practices are likely stronger in higher impact factor journals. Our findings highlight the need for journals and the STROBE guideline to provide more explicit guidance on the requirements for confounder selection, which could help improve reporting practices.
