Creation and validation of tool to assess resident competence in neonatal resuscitation

Abstract

Background The American Board of Pediatrics requires that pediatricians be able to initiate stabilization of a newborn. After residency, 45% of general pediatricians routinely attend deliveries. However, there is no standard approach or tool to measure resident proficiency in newborn resuscitation across training programs. In a national survey, we found a large variability in faculty assessment of the amount of supervision trainees need for various resuscitation scenarios. Objective documentation of trainee performance would permit competency-based decisions on the level of supervision required and facilitate feedback on trainee performance.

Methods A simplified tool was created following the Neonatal Resuscitation Program (NRP) algorithm, with emphasis on communication, leadership, knowledge of equipment, and initial stabilization. To achieve content validity, the tool was evaluated by the NRP steering committee. To assess internal structure of the tool, we filmed 10 simulated resuscitation scenarios 9 of which contained errors. Experienced resuscitation team members used the tool to assess performance of the team leader in the videos. To evaluate the response process, the tool was used to assess experienced resuscitators in real time at academic and non-academic sites.

Results The NRP steering committee approved the tool, providing evidence of content validity. Performance of the team leader in the simulated videos was assessed by 16 evaluators using the tool. There was an intra-class coefficient of 0.86 showing excellent agreement. There was no statistical difference in scores between 102 resuscitations led by experienced resuscitators at academic and non-academic hospitals (p=0.98), which demonstrates generalizability.

Conclusions The tool we have developed to assess performance in initiating newborn resuscitation shows evidence of construct validity based on assessment of content and internal structure (inter-observer agreement, response processes, and generalizability).