Research Letter
Randomized Experiment of Recruitment Letter Design to Maximize Clinical Trial Enrollment
Niklas Dyrby Johansen, David Preiss, Marion Mafham, et al
JAMA Published Online: June 25, 2025
doi: 10.1001/jama.2025.9170
Most new medical therapies only offer small to moderate benefits above high-level standard care and require increasingly large trials to demonstrate efficacy in light of generally declining incidence rates of major clinical outcomes.1,2 Cost-effective strategies to improve trial recruitment are therefore urgently needed. Prior investigations have yielded mixed results, and those assessing the effects of modifying written recruitment material have often been limited by small sample sizes and high risk of bias.3,4 Therefore, within the context of the DANFLU-2 (A Pragmatic Randomized Trial to Evaluate the Effectiveness of High-Dose Quadrivalent Influenza Vaccine vs Standard-Dose Quadrivalent Influenza Vaccine in Older Adults) trial, a randomized experiment was conducted to investigate whether varying digital recruitment letter layout and color scheme would impact trial enrollment.
Methods
DANFLU-2 (NCT05517174) is an ongoing pragmatic randomized trial of high-dose inactivated influenza vaccine vs standard-dose inactivated influenza vaccine in adults 65 years or older in Denmark,5 conducted under approval from the Medical Research Ethics Committees in Denmark and the Danish Medicines Agency (EU CT number: 2022-500657-17-00). During the recruitment period of the trial’s second influenza season (2023-2024), as a study within a trial, we conducted a randomized experiment of digital recruitment letter strategies in which we varied layout and color scheme in a 3 × 3 factorial design. Invitees were randomized in a 1:1:1 ratio to receive 1 of 3 layout variants (single-page letter, 2-page letter, or enhanced 2-page letter with additional colored text boxes emphasizing certain elements) and also in a 1:1:1 ratio to receive 1 of 3 color schemes (dark red, blue, or green) (Figure 1). Danish citizens who opted for physical letters instead of email were excluded. Text content was identical across all letters. All letters included a colored box with a link to the sign-up website. English translations of the letters can be found in the eMethods in Supplement 1. The recruitment letters were delivered through the Danish governmental electronic mailbox system. The primary end point of this analysis was enrollment into DANFLU-2. Two-sided P values <.05 were considered statistically significant. This report follows the Consolidated Standards of Reporting Trials (CONSORT) reporting guideline.
A total of 934 049 Danish citizens ≥65 years were identified and randomized to receive different letter types. The mean (SD) age of participants was 74.4 (6.4) years and 484 589 (51.9%) were female. Letters were successfully delivered to 99.95% of those randomized. DANFLU-2 enrolled a total of 160 451 participants during the 2023-2024 season. Enrollment was significantly more likely among participants who received the single-page letter compared with the 2-page letter (16.28% vs 16.02%; difference, 0.25 [95% CI, 0.07-0.44] percentage points; P = .007) and the enhanced 2-page letter (16.28% vs 15.29%; difference, 0.98 [95% CI, 0.80-1.16] percentage points; P < .001) (Figure 2). The 2-page letter was more effective than the enhanced 2-page letter. No differences were observed across color schemes. No interactions were observed between letter layout and color scheme (all P values for interaction >.75). Compared with the enhanced 2-page letter, the effect estimates observed with both the 1-page letter and the standard 2-page letter were significantly greater among males compared with females as well as among younger compared with older individuals (Figure 2).
Comparisons were analyzed using the intention-to-treat principle. No adjustments were made for multiple comparisons. The median age of the study population was 73.7 years.
aThe P values for letter type and color scheme are P values for effect and were derived from χ2 tests. Interaction P values were calculated on the absolute scale using binomial regression with interaction terms.
As the largest individually randomized influenza vaccine trial ever conducted, DANFLU-2 offered a unique opportunity to test the effects of different invitation letter design features on trial enrollment on a very large scale. The results of the current study suggest a preference for shorter-form messaging with differences of approximately 1 percentage point in absolute terms and approximately 6% in relative terms. Consequently, compared with the least effective letter (the enhanced 2-page letter), sending the most effective letter (the 1-page letter) to 1 000 000 potential participants would result in an additional approximately 9800 trial participants, thereby increasing the probability of reaching the planned sample size and securing sufficient statistical power. The potential heterogeneity observed according to age and sex suggests potential additional benefit from further tailored strategies. Although the results may not be directly generalizable to trials with other designs or conducted in different settings, these findings do underline the importance of carefully considering design and style of participant-facing material in clinical trials, including recruitment letters, particularly in decentralized pragmatic trials where broadly distributed, low-touch recruitment strategies are often necessary.
