Invited Commentary
Medical Education
January 9, 2025
Another Resident Survey—Useful to Whom?
Julie L. Oyler, Jason T. Alexande
JAMA Netw Open. 2025;8(1):e2454054. doi:10.1001/jamanetworkopen.2024.54054
Grbic and colleagues1 performed a cross-sectional study examining whether program director (PD) assessments of postgraduate year 1 (PGY-1) resident performance during the transition to residency differed by specialty. This large, retrospective cross-sectional study addressed an important goal of the 2021 Coalition for Physician Accountability Undergraduate-Graduate Review Committee report2 and found that while the vast majority of PGY-1 residents met expectations of PDs (96.8%), variation occurred among specialty programs.
The Association of American Medical Colleges Resident Readiness Survey (RRS)3,4 is a 20-question survey completed by PDs of matched PGY-1 residents from participating medical schools after 6 months of clinical experience. The RRS was intended to standardize medical school surveys sent to PDs that historically could be medical school specific, specialty specific, or both. Such an intervention would theoretically decrease PD effort in evaluating PGY-1 readiness by improving consistency and familiarity with a standard evaluation form while affording medical schools the opportunity to review the readiness of their graduates for residency across specialties.
Grbic and colleagues1 demonstrated the feasibility of the RRS: more than twice as many medical schools participated in 2022 than did in 2020, and PDs completed more than 50% more surveys in the third year of the RRS than they did at its inception. These results alone represent a positive step forward in creating a shared evaluation platform for medical training.5 PGY-1 residents who matched into university- or community-based programs from multiple specialties were represented in the analysis of the RRS, allowing for some meaningful comparisons, particularly among the PGY-1 residents who did not meet expectations.
First, PGY-1 residents had higher odds of not meeting overall expectations in the second and third year of the survey compared with the initial graduating class. Second, odds of not meeting expectations varied by specialty. Using PGY-1 residents who matched into internal medicine as a reference, PGY-1 residents who matched into family medicine, obstetrics and gynecology, or general surgery had higher odds of not meeting overall expectations. Third, PGY-1 residents who stayed at their home institution had higher odds of being rated as meeting or exceeding expectations when compared with PGY-1 residents who completed their medical training elsewhere.
While it is incorrect to determine causality with retrospective data, attempting to explain differences in data are what we do as educators and clinicians. Ignoring the impact of the COVID-19 pandemic on clinical training of medical students seems shortsighted. The higher odds of PGY-1 residents not meeting overall expectations at 6 months of training among the 2021 and 2022 graduating classes compared with the 2020 class makes logical sense and seems to support the experiences of many PDs.6 With respect to comparisons of PGY-1 residents not meeting overall expectations between specialties, Grbic and colleagues1 concluded that specialty specific variation was not attributable to competitiveness of the specialty since “a mix of specialty categories that were more competitive for entry into GME [graduate medical education], . . . such as obstetrics and gynecology, and that were less competitive for entry into GME, such as family medicine, were among those associated with higher odds of not meeting overall expectations.” Alternatively, we argue that specialties with higher levels of diversity of clinical experience (namely family medicine and obstetrics and gynecology) pose a greater challenge to incoming PGY-1 residents and thus would have higher odds of not meeting overall expectations.
The authors also argue that the higher odds of PGY-1 residents being rated as meeting or exceeding overall expectations among residents who matched into a specialty at the same location that they did their undergraduate medical training demonstrates a “home-field advantage.” Understanding this phenomenon in greater detail would be profoundly helpful. Are congruence of applicant and program preferences and priorities, as the authors suggest, the likely explanation for this association, or rather, are PGY-1 residents who have matched at their home institutions more likely to be on more difficult rotations (eg, rotations in the intensive care unit) in the first few months of their PGY-1, when expectations are lower and are thus easier to exceed?
Regardless of the veracity of any of these arguments, the RRS gives graduate and undergraduate medical education programs the opportunity to improve their ability to shepherd learners along the continuum of medical education. However, who benefits from the RRS? The sharp yearly increase in medical school participation in the RRS observed in this study is not terribly surprising. Standardized assessment of the readiness of past graduates regardless of matched specialty provides an outstanding opportunity for medical schools to identify ways to revise curricula and career counseling in the preclinical and clinical years.
On the other hand, what benefits do PDs stand to gain? While it is conceivable that the results of the surveys that they have spent time completing may benefit their programs in the future, longer-term data will be needed to assess this potential effect. As 2 authors who collectively have nearly 30 years of reading residency applications, we can attest that the information medical schools provide to programs during the residency selection process is becoming more opaque. Application volume continues to increase; clerkship grades and licensing exams are increasingly becoming pass or fail; reporting of information from medical schools to residency programs remains highly variable; and while the RRS may prove valuable to medical schools, PDs are not out of place for expecting additional transparency from medical schools in return.
And what of the trainees themselves? While 96.8% of medical graduates met or exceeded overall expectations of their PDs in their matched specialty, the 3.2% of PGY-1 residents who did not meet expectations have still gone through at least 4 years of medical school training, likely incurred substantial debt,7 and are yet inadequately prepared for their PGY-1 as judged by their PDs. Is bias by the PDs or clinical competency committees playing a role in assessing whether PGY-1 residents are meeting expectations? For example, do odds of not meeting expectations vary between differences or congruence in demographic data between medical schools, PDs, programs, and PGY-1 residents? Furthermore, what percentage of PGY-1 residents not meeting expectations as identified by the RRS receive enough training in residency to become successful physicians and at what cost? As the RRS continues to be used by PDs and medical schools, the accrual of more longitudinal data will help address some of these questions.
Grbic and colleagues1 should be commended for their work in studying the RRS across multiple specialties and program settings. In part, their work demonstrates that standardizing assessment of PGY-1 residents’ readiness of practice is feasible and provides an opportunity to improve transparency in the continuum of medical training. Whether the RRS endures as part of the undergraduate to graduate medical education transition rests in part on its ability to positively impact all stakeholders. Medical schools currently benefit the most; how much residency programs and future trainees may benefit remains to be seen.
