Research Letter
Continuous Medical Knowledge Assessment Progress Report Score and Hospitalized Patient Outcomes
Bradley M. Gray, Jonathan L. Vandergrift, Rebecca S. Lipner, et al
JAMA Intern Med Published Online: June 16, 2025
doi: 10.1001/jamainternmed.2025.1730
The American Board of Internal Medicine started offering a Longitudinal Knowledge Assessment (LKA) in 2022 as an alternative to the 10-year long-form Maintenance of Certification assessment. Participants answer 30 questions in an open-book format quarterly, and competency is assessed every 5 years. Physicians receive preliminary scores to identify knowledge gaps, but how these are associated with clinical outcomes is unclear. We measure associations between the first-year LKA score and hospitalized patient outcomes for the first 2 cohorts of participating hospitalists.
Our cross-sectional study, conducted among hospitalists enrolled in LKA in 2022 and 2023, examined associations between preliminary scores (hereafter, scores) and hospitalized patient outcomes 2 years prior to enrollment. The sample included hospital stays for Medicare fee-for-service beneficiaries older than 65 years admitted for nonelective common diagnoses to hospitals with at least 100 beds (Figure 1; eMethods 1 in Supplement 1). Hospitalizations were attributed to physicians with a plurality of hospitalization’s evaluation and management contacts who saw the patient within 3 days of admission.1,2 The study was deemed exempt from review and consent by the Advarra institutional review board because data were deidentified. The STROBE reporting guideline was followed.
Figure 1. Flow of Physicians and Patient Hospitalizations in the Study
DRG indicates diagnosis-related group; ESKD, end-stage kidney disease; LKA, Longitudinal Knowledge Assessment; and NPI, National Provider Identifier.
aDefined as more than 100 total evaluation and management contacts, with more than 90% being inpatient contacts.
bHospitals with 100 beds or less were excluded to accommodate hospital fixed effects.
cIncluded only common DRG codes (eMethods 1 and eTable in Supplement 1).
dHospitalizations attributed to physicians with plurality of evaluation and management contacts during that hospitalization among generalist physicians with at least 1 contact within 3 days of admission.
eExcludes 562 hospitalizations because admission date was fewer than 7 days before end of 2022.
fExcludes 936 hospitalizations because discharge date was fewer than 7 days before end of 2022.
gExcludes 2 physicians and 3418 hospitalizations because admission date was fewer than 30 days before end of 2022.
hExcludes 2 physicians and 4024 hospitalizations because discharge date was fewer than 30 days before end of 2022.
Primary outcomes were 7-day postadmission mortality and 7-day readmission.3 Secondary measures included outcomes at 30 days, length of stay (LOS), and consultation frequency.2 The score measure was the end-of-first-year score quartile (cutoffs based on all participants).
Our statistical approach leveraged the pseudorandom assignment of patients to hospitalists within hospitals by estimating associations using multivariable linear hospital fixed-effects regression, which compares scores and outcomes among physicians within the same hospital.4 We controlled for patient demographics, risk factors, time of year, and physician enrollment year1 ,2,5 (eMethods 2 and 3 in Supplement 1). A 2-sided P < .05 was considered significant.
Our study included 4064 physicians (mean [SD] age, 45.9 [8.4] years; 64.9% men) providing care for 261 225 hospitalizations (Figure 1). Per 1000 hospitalizations, the 7-day unadjusted mortality was 50.3, and the readmission rate was 57.6.
Per 1000 hospitalizations, the 7-day adjusted mortality difference for physicians in the top vs bottom quartile was −4.1 (95% CI, −7.7 to −0.5; P = .03), a 7.8% difference (Figure 2). This difference was similar when comparing physicians in the second (−3.8 [95% CI, −7.1 to −0.6]; P = .02) or third quartile (−3.9 [95% CI, −7.1 to −0.6]; P = .02) with those in the bottom quartile.
Figure 2. Preliminary Score Quartile Associations: Adjusted Percentage Difference Compared With the Bottom Quartile Mean Outcome
Score quartiles are based on the Longitudinal Knowledge Assessment (LKA) end-of-first-year progress report score ranking. NA indicates not applicable.
aAdjusted for hospital fixed effects; LKA enrollment year; physician years of experience; hospital day; and patient race and ethnicity, age, sex, Medicaid eligibility, zip code (rural), median income, US Department of Health and Human Services (HHS) region, Elixhauser Comorbidity Index score and indicators, diagnosis-related group code, weekend, season, and season interaction with year and HHS region (eMethods 3 in Supplement 1).
bPercentage difference equals the difference between the quartile-adjusted mean outcome minus the first-quartile adjusted mean outcome divided by the adjusted first-quartile outcome mean, holding other characteristics constant (eMethods 3 in Supplement 1).
Per 1000 hospitalizations, the 7-day adjusted readmission difference for physicians in the top vs bottom quartile was −3.1 (95% CI −6.2 to −0.1; P = .046). This difference was similar when comparing physicians in the second vs bottom quartile. We observed a significantly higher mortality among physicians in the bottom vs all each other quartiles. Adjusted differences between quartiles were not significant for 30-day readmission, LOS, or consultations.
Seven-day mortality and readmissions were lower for physicians in the top vs bottom quartile of LKA scores, consistent with prior research that found similar associations with initial certification scores.1,6 Similar associations were observed between the bottom and second or third quartiles, suggesting preliminary scores may best identify physicians at the low end of quality. Similar associations were seen for 30-day mortality. Associations with 30-day readmissions, LOS, and consultations were not significant.
Our study has limitations. Although we can attribute patient outcomes to physicians due to quasi-random, within-hospital patient assignment among hospitalists, we cannot assess whether associations with score quartile were due to medical knowledge differences or other physician characteristics correlated with scores. Outcomes were assessed prior to the LKA, so associations reflect the care provided by physicians who later performed well or poorly on the LKA but do not reflect changes in knowledge from LKA participation. Our results may not generalize to other settings, LKA cohorts, or the final LKA score. In summary, hospitalized patients cared for by physicians with high vs low preliminary LKA scores experienced better outcomes.
