Browsing by Subject "Critical Appraisal"

Now showing 1 - 3 of 3
  • Thumbnail Image
    Item
  • Thumbnail Image
    Item
    How Do Medical Students Approach Critical Appraisal? Results from a Mixed-Methods Study
    (2023-05-19) Menard, Laura M.; Blevins, Amy E.; Vetter, Cecelia J.; Trujillo, Daniel J.
    OBJECTIVES: Our research team wanted to find out what principles and best practices medical students use when prompted by a clerkship assignment to complete a critical appraisal of an article of their choosing. Our hypothesis was that, outside of a structured classroom environment, many students would default to more basic literature evaluation strategies or even apply proxies for methodological rigor such as journal reputation or peer review status of a study. METHODS: All first-year clerkship students at the School of Medicine are required to complete a patient-focused evidence-based medicine (EBM) assignment during their Internal Medicine clerkship. A team of three librarians and one statistician undertook a mixed-methods approach to identify and quantify themes that emerged in the text of one year's worth of these assignments (n=343). A mixed method research approach was implemented to gain a greater understanding of the EBM principles and best practices that students reference in their assignments. Within this approach a qualitative content analysis was conducted, followed by a quantitative analysis of patterns within the sample. The research team used first- and second cycle coding and a collaboratively developed code list of nine major codes to ensure accuracy and standardization. Additionally, the research team's statistician implemented an inter-rater reliability plan and examined inter-class correlations to ensure grading consistency across team members and student assignments. Once all assignments had been coded, the team used statistical analysis to find correlations between codes as well as frequency of code application within the sample in order to identify five major critical appraisal themes which emerged in the students' assignments. RESULTS: After a rigorous coding process, several codes and related themes emerged. The research team identified nine main codes and five major themes. These themes are as follows: Theme 1: Comparing the study population to the patient being treated and recommending a course of action Theme 2: Identifying study type and position in hierarchy of evidence Theme 3: Identifying proxies for study quality, including provenance and timeliness of chosen study Theme 4: Summarizing study methodology and results Theme 5: Attempting a critical appraisal of chosen study Additionally, we identified correlations between themes as well as frequency of application in the sample. CONCLUSIONS: A few notable results from our analysis of this sample are the frequency with which students were able to summarize the results of their chosen study and apply what they had learned to patient care (54.5% and 46.9% of all assignments, respectively). However, a notable number of students (35.2%) incorrectly used journal reputation, peer review status, h-index, impact factor, or similar metric, as a proxy for critical appraisal without engaging with the study methodology. This indicates that there is a need for further education and engagement with clerkship directors regarding the utility and application of EBM skills in the clinical curriculum.
  • Thumbnail Image
    Item
    It’s Not the Upside Down: Creating a Flipped Classroom Experience for Critical Appraisal
    (2019-10-06) Menard, Laura M.; Blevins, Amy E.
    OBJECTIVE: Critically appraising medical literature is a skill that every medical student needs, however, finding experienced instructors and time in the curriculum can be challenging – especially with multiple campuses. An assistant director worked with an associate director to design a flipped classroom model for teaching critical appraisal skills to medical students. In addition, the assistant director designed an in-house training program and facilitator guides to ensure that librarians had the skills to deliver the sessions to campuses around the state. METHODS: Using Kaltura, the assistant director designed and recorded short video tutorials with embedded formative assessments for prognosis, harm, and diagnosis. Librarians attended a “teach through” where they were introduced to the material. Students completed the video tutorials and a summative quiz within a course management system ahead of a facilitated small group session. The summative quiz included questions to assess satisfaction and comprehension. All librarians reviewed the summative quiz data to customize the sessions. RESULTS: Summative quiz feedback was positive. The data provided by the embedded formative assessments showed that students were able to answer questions based on the information provided in the tutorials. In addition, anecdotal evidence from the course director shows that students were satisfied with the new modality. Librarians used both the summative and formative quiz results to tailor the sessions to individual campus needs. This also allowed for scaffolding and reinforcement of instruction across the three sessions. CONCLUSIONS: The flipped classroom model allowed librarians to maximize critical appraisal instruction while minimizing face-to-face classroom time. In addition, librarians felt comfortable serving as facilitators rather than lecturers for material that librarians had not previously taught. We will continue to use this model since it provides the students with self-directed learning opportunities with minimal disruption to course and librarian schedules.