Integrating evidence-based medicine skills into a medical school curriculum: a quantitative outcomes assessment

Abstract

Objectives: This research project aims to determine the potential differential impact of two curricular approaches to teaching evidence-based medicine (EBM) on student performance on an EBM assignment administered during the first year of clerkship. A meaningful result would be any statistically significant difference in scores on the assignment given to measure student performance. Design: In order to assess and compare student learning under the different curricula, the PI and a team of five faculty members blinded to assignment date and other possibly identifying details used a modified version of the previously validated Fresno rubric to retrospectively grade three years’ worth of EBM assignments given to students in clerkship rotations 1-3 (n = 481) during the Internal Medicine clerkship. Specifically, EBM performance in three separate student cohorts were examined. Setting: The study took place at a large Midwestern medical school with nine campuses across the state of Indiana. Participants: Study participants were four hundred eighty-one students who attended the medical school and completed the Internal Medicine clerkship between 2017-2019. Interventions: Prior to the inception of this study, our institution had been teaching EBM within a discrete 2-month time period during medical students’ first year. During a large-scale curricular overhaul, the approach to teaching EBM was changed to a more scaffolded, integrated approach with sessions being taught over the course of two years. In this study, we assess the differential impact of these two approaches to teaching EBM in the first two years of medical school. Main Outcome Measures: We used clerkship-level EBM assignment grades to determine whether there was a difference in performance between those students who experienced the old versus the new instructional model. Clerkship EBM assignments given to the students used identical questions each year in order to have a valid basis for comparison. Additionally, we analyzed average student grades across the school on the EBM portion of Step 1. Results: Four hundred and eighty-one assignments were graded. Mean scores were compared for individual questions and cumulative scores using a one-way Welch ANOVA test. Overall, students performed .99 of a point better on the assignment from Year One (Y1), prior to EBM curriculum integration, to Year Three (Y3), subsequent to EBM integration (p= <.001). Statistically significant improvement was seen on questions measuring students’ ability to formulate a clinical question and critically appraise medical evidence. Additionally, on USMLE Step 1, we found that student scores on the EBM portion of the exam improved from Y1 to Y3. Conclusions: Results of this study suggest that taking a scaffolded, curriculum-integrated approach to EBM instruction during the pre-clinical years increases, or at the very least does not lessen, student retention of and ability to apply EBM concepts to patient care. Although it is difficult to fully attribute students’ retention and application of EBM concepts to the adoption of a curricular model focused on scaffolding and integration, the results of this study show that there are value-added educational effects to teaching EBM in this new format. Overall, this study provides a foundation for new research and practice seeking to improve EBM instruction. Trial Registration: IRB approval (Protocol #1907054875) was obtained for this study.