Guide to Statistics and Methods 

Surgical Education Research

January 3, 2024

Practical Guide to Ethics in Surgical Education Research

Michael M. Awad, Amy H. Kaji, Timothy M. Pawlik

JAMA Surg. Published online January 3, 2024. doi:10.1001/jamasurg.2023.6705

Introduction

Rigorous and well-designed research is integral to advance surgical education science. In contrast to clinical research in which study participants are patients, surgical education research has learners as the focus. Learners are a vulnerable population,¹ and it is important that education research adheres to the ethical principles for medical research involving human participants (Box).²

Box. 

Summary

Several unique ethical considerations are key to the safe and appropriate conduct of surgical education research.

• Learners are considered a vulnerable human participant study population.
• In contrast to program evaluations and learner assessment, if analysis of learners is to be disseminated, it is considered research and has additional ethical requirements.
• Surgical education researchers should pay special attention to issues around consent, confidentiality, and incentivization unique to learner study participants.
• Lack of attention to ethics in the design and conduct of a surgical education study can lead to learner disengagement, distrust, and severe medicolegal consequences.

Using the Methodology

When and Why This Method Might Be Used

There are many overlapping goals, such as evaluation, assessment, and research, each with its own ethical considerations. Program evaluation is generally conducted internally to determine whether learners as a group meet the stated goals for the purpose of iterative program quality improvement.³ Learner assessment aims to determine whether individuals achieved specific objectives for the purpose of formative feedback or for higher-stakes summative assessment. When the intent of the program intervention or learner assessment is dissemination of information through presentations or publications, this activity is considered research and is subject to ethical research guidelines. Generalizability of research findings are an ethical issue because any evidence should be applicable to a broad population to maximize the research benefit.

How This Methodology Should Be Used

Consent

There are several important considerations involving consent of learners in an educational research project. Will learners feel pressure to participate? What happens when a learner refuses? Trainees may fear reprisal if they choose not to participate in a research project, especially if asked by someone in a supervisory role.

To mitigate these concerns, the member of the study team obtaining consent should be someone who is not an evaluator or in a position of program authority.⁴ Record of participation should be kept confidential. If the research involves retrospective review of anonymous data, the need for informed consent is unlikely; however, the research should still undergo review by a human participants review board. If the educational intervention involves clinical outcomes, consideration should be given to consenting patients in addition to the learner.

Performance Evaluation

If learner assessment is part of a research study, learner performance should generally not be assessed for other purposes such as remediation, advancement, or graduation. As the outcome of the study question is presumably unknown, the data should not be used to make high-stakes decisions. When high-stakes decisions are part of the research question, the study team should not include individuals making those decisions. If such an arrangement is not possible, the research study should be performed after the relevant decisions are made.

Confidentiality

The sample population of interest in many surgical education research projects is often small, as surgical clerkships, residency classes, and fellowship programs often have few individuals in each cohort. Subgroup analyses by postgraduate year, race and ethnicity, or gender can inadvertently identify individuals. As such, confidentiality can be a significant concern in surgical education research, perhaps even more so than in other types of research.

To ensure learner confidentiality, several strategies should be used. Ideally, data should be anonymized and blinded. A study team member who is not involved in learner evaluation or promotion decisions should store the data in a secure, password-protected location. True anonymization may not be possible due to the nature of the study question or the need to follow up with individuals longitudinally before and after an intervention. In these cases, data should be made confidential by assigning a unique identifier to each study participant that is known only to study personnel removed from evaluation responsibilities. Confidentiality and research quality can be strengthened by increasing the study population size by pooling multiple learner groups across years or even across institutions.

Study Incentives

Incentives are often offered to study populations to encourage participant recruitment and completion of study milestones. In surgical education research, outright cash payments of any substantive value directly to learners should be avoided. Many learners incur substantial financial debt as part of their medical training, and monetary compensation may be considered enticement to participate.

Incentives for learner study participants should be of nominal value, such as a small, noncash gift card. Rewards toward course credit, better call shifts, or vacation days should be strictly avoided.⁵

Resources Required for Using the Method

To support the ethical conduct of surgical education research, several best practices should be observed. Study personnel should not be directly involved in the evaluation or advancement decisions of learners. A research coordinator or staff member may be involved in the recruitment process, obtaining consent, anonymization of data, maintaining the secure database, and data analysis; a statistician or methodology expert should be included from study inception.

All studies involving human participants must be reviewed by a human participants review committee or equivalent. Some institutions may also use an educational research committee or educational ethics committee.¹ These committees recognize special considerations of educational research initiatives and learner participants, as standard review boards often deal solely with clinical research and may be unfamiliar with education research concepts.

Advantages and Limitations of the Method

Adherence to ethical principles can help ease learner concerns about participation in research studies and encourage recruitment and retention. Planning and implementation of an ethically sound research program from the outset can help facilitate review by an institutional review board. In contrast, lack of adherence to sound ethical and methodologic guidelines can lead to legal challenges that can be devastating to an institution and the individuals involved. While including ethical considerations may take some additional time and resources by the study team, there are no significant downsides to the ethical conduct of surgical education research.

Statistical Considerations

For any study type, published reporting guidelines should be followed to optimize research quality and transparency.⁶ Equipoise in the research question is necessary for adherence to ethical principles. Participant selection must be based on the principles of equity, diversity, and inclusion to best represent the target population for which the research question applies. If an educational intervention is meant for all levels of learners, its performance should be evaluated and described accounting for differences in experience, knowledge, and other factors. On occasion, ethical or practical reasons may preclude random selection or a true experimental study design. In these circumstances, a quasi-experimental approach should be considered.

Selection bias occurs when individuals or groups for study comparison differ systematically for known confounders or baseline characteristics with respect to the outcome of interest (eg, attrition) and lead to faulty conclusions. Randomized assignment (when feasible) is a possible design approach to avoid these issues. If regression analysis is performed, it is important to preidentify known confounders and adjust for any differences in the analysis plan. If an assessment is being performed longitudinally across individuals, consider within-learner correlations in performance and within-evaluator correlations of assessment. Individuals assessing the learner’s performance should ideally be blinded to the participant’s identity. If there are large amounts of missing data, consideration should be given to adjustment through imputation. Detailed study design including fastidious data capture and auditing and collecting limited essential data fields are some approaches to minimize missing data. Conclusions are only applicable to the specific types of individuals included in the study.

Where to Find More Information

Other information on this topic is available. Recommended sources are Boileau et al⁷ and Klitzman.⁸