- Browse by Subject
Browsing by Subject "Educational measurement"
Now showing 1 - 5 of 5
Results Per Page
Sort Options
Item Anatomy Nights: An international public engagement event increases audience knowledge of brain anatomy(PLOS, 2022-06-09) Sanders, Katherine A.; Philp, Janet A.C.; Jordan, Crispin Y.; Cale, Andrew S.; Cunningham, Claire L.; Organ, Jason M.; Anatomy, Cell Biology and Physiology, School of Medicine; Anatomy, Cell Biology and Physiology, School of MedicineAnatomy Nights is an international public engagement event created to bring anatomy and anatomists back to public spaces with the goal of increasing the public's understanding of their own anatomy by comparison with non-human tissues. The event consists of a 30-minute mini-lecture on the anatomy of a specific anatomical organ followed by a dissection of animal tissues to demonstrate the same organ anatomy. Before and after the lecture and dissection, participants complete research surveys designed to assess prior knowledge and knowledge gained as a result of participation in the event, respectively. This study reports the results of Anatomy Nights brain events held at four different venues in the UK and USA in 2018 and 2019. Two general questions were asked of the data: 1) Do participant post-event test scores differ from pre-event scores; and 2) Are there differences in participant scores based on location, educational background, and career. We addressed these questions using a combination of generalized linear models (R's glm function; R version 4.1.0 [R Core Team, 2014]) that assumed a binomial distribution and implemented a logit link function, as well as likelihood estimates to compare models. Survey data from 91 participants indicate that scores improve on post-event tests compared to pre-event tests, and these results hold irrespective of location, educational background, and career. In the pre-event tests, participants performed well on naming structures with an English name (frontal lobe and brainstem), and showed signs of improvement on other anatomical names in the post-test. Despite this improvement in knowledge, we found no evidence that participation in Anatomy Nights improved participants' ability to apply this knowledge to neuroanatomical contexts (e.g., stroke).Item Assessing the Transition of Training in Health Systems Science From Undergraduate to Graduate Medical Education(Accreditation Council for Graduate Medical Education, 2021) Santen, Sally A.; Hamstra, Stanley J.; Yamazaki, Kenji; Gonzalo, Jed; Lomis, Kim; Allen, Bradley; Lawson, Luan; Holmboe, Eric S.; Triola, Marc; George, Paul; Gorman, Paul N.; Skochelak, Susan; Medicine, School of MedicineBackground: The American Medical Association Accelerating Change in Medical Education (AMA-ACE) consortium proposes that medical schools include a new 3-pillar model incorporating health systems science (HSS) and basic and clinical sciences. One of the goals of AMA-ACE was to support HSS curricular innovation to improve residency preparation. Objective: This study evaluates the effectiveness of HSS curricula by using a large dataset to link medical school graduates to internship Milestones through collaboration with the Accreditation Council for Graduate Medical Education (ACGME). Methods: ACGME subcompetencies related to the schools' HSS curricula were identified for internal medicine, emergency medicine, family medicine, obstetrics and gynecology (OB/GYN), pediatrics, and surgery. Analysis compared Milestone ratings of ACE school graduates to non-ACE graduates at 6 and 12 months using generalized estimating equation models. Results: At 6 months both groups demonstrated similar HSS-related levels of Milestone performance on the selected ACGME competencies. At 1 year, ACE graduates in OB/GYN scored minimally higher on 2 systems-based practice (SBP) subcompetencies compared to non-ACE school graduates: SBP01 (1.96 vs 1.82, 95% CI 0.03-0.24) and SBP02 (1.87 vs 1.79, 95% CI 0.01-0.16). In internal medicine, ACE graduates scored minimally higher on 3 HSS-related subcompetencies: SBP01 (2.19 vs 2.05, 95% CI 0.04-0.26), PBLI01 (2.13 vs 2.01; 95% CI 0.01-0.24), and PBLI04 (2.05 vs 1.93; 95% CI 0.03-0.21). For the other specialties examined, there were no significant differences between groups. Conclusions: Graduates from schools with training in HSS had similar Milestone ratings for most subcompetencies and very small differences in Milestone ratings for only 5 subcompetencies across 6 specialties at 1 year, compared to graduates from non-ACE schools. These differences are likely not educationally meaningful.Item A Critical Disconnect: Residency Selection Factors Lack Correlation With Intern Performance(Accreditation Council for Graduate Medical Education, 2020) Burkhardt, John C.; Parekh, Kendra P.; Gallahue, Fiona E.; London, Kory S.; Edens, Mary A.; Humbert, A.J.; Pillow, M. Tyson; Santen, Sally A.; Hopson, Laura R.; Emergency Medicine, School of MedicineBackground: Emergency medicine (EM) residency programs want to employ a selection process that will rank best possible applicants for admission into the specialty. Objective: We tested if application data are associated with resident performance using EM milestone assessments. We hypothesized that a weak correlation would exist between some selection factors and milestone outcomes. Methods: Utilizing data from 5 collaborating residency programs, a secondary analysis was performed on residents trained from 2013 to 2018. Factors in the model were gender, underrepresented in medicine status, United States Medical Licensing Examination Step 1 and 2 Clinical Knowledge (CK), Alpha Omega Alpha (AOA), grades (EM, medicine, surgery, pediatrics), advanced degree, Standardized Letter of Evaluation global assessment, rank list position, and controls for year assessed and program. The primary outcomes were milestone level achieved in the core competencies. Multivariate linear regression models were fitted for each of the 23 competencies with comparisons made between each model's results. Results: For the most part, academic performance in medical school (Step 1, 2 CK, grades, AOA) was not associated with residency clinical performance on milestones. Isolated correlations were found between specific milestones (eg, higher surgical grade increased wound care score), but most had no correlation with residency performance. Conclusions: Our study did not find consistent, meaningful correlations between the most common selection factors and milestones at any point in training. This may indicate our current selection process cannot consistently identify the medical students who are most likely to be high performers as residents.Item Measurement of Nontechnical Skills During Robotic-Assisted Surgery Using Sensor-Based Communication and Proximity Metrics(American Medical Association, 2021-11-01) Cha, Jackie S.; Athanasiadis, Dimitrios; Anton, Nicholas E.; Stefanidis, Dimitrios; Yu, Denny; Surgery, School of MedicineThis cohort study uses sensor-based communication and proximity metrics to assess surgeon nontechnical skills during robotic-assisted surgery.Item Tools to Assess Behavioral and Social Science Competencies in Medical Education: A Systematic Review(Wolters Kluwer, 2016-05) Carney, Patricia A.; Palmer, Ryan T.; Miller, Marissa Fuqua; Thayer, Erin K.; Estroff, Sue E.; Litzelman, Debra K.; Biagioli, Frances E.; Teal, Cayla R.; Lambros, Ann; Hatt, William J.; Satterfield, Jason M.; Medicine, School of MedicinePURPOSE: Behavioral and social science (BSS) competencies are needed to provide quality health care, but psychometrically validated measures to assess these competencies are difficult to find. Moreover, they have not been mapped to existing frameworks, like those from the Liaison Committee on Medical Education (LCME) and Accreditation Council for Graduate Medical Education (ACGME). This systematic review aimed to identify and evaluate the quality of assessment tools used to measure BSS competencies. METHOD: The authors searched the literature published between January 2002 and March 2014 for articles reporting psychometric or other validity/reliability testing, using OVID, CINAHL, PubMed, ERIC, Research and Development Resource Base, SOCIOFILE, and PsycINFO. They reviewed 5,104 potentially relevant titles and abstracts. To guide their review, they mapped BSS competencies to existing LCME and ACGME frameworks. The final included articles fell into three categories: instrument development, which were of the highest quality; educational research, which were of the second highest quality; and curriculum evaluation, which were of lower quality. RESULTS: Of the 114 included articles, 33 (29%) yielded strong evidence supporting tools to assess communication skills, cultural competence, empathy/compassion, behavioral health counseling, professionalism, and teamwork. Sixty-two (54%) articles yielded moderate evidence and 19 (17%) weak evidence. Articles mapped to all LCME standards and ACGME core competencies; the most common was communication skills. CONCLUSIONS: These findings serve as a valuable resource for medical educators and researchers. More rigorous measurement validation and testing and more robust study designs are needed to understand how educational strategies contribute to BSS competency development.