Gichoya, Judy W.Sinha, PriyanshuDavis, MelissaDunkle, Jeffrey W.Hamlin, Scott A.Herr, Keith D.Hoff, Carrie N.Letter, Haley P.McAdams, Christopher R.Puthoff, Gregory D.Smith, Kevin L.Steenburg, Scott D.Banerjee, ImonTrivedi, Hari2022-02-182022-02-182022-02Gichoya, J. W., Sinha, P., Davis, M., Dunkle, J. W., Hamlin, S. A., Herr, K. D., Hoff, C. N., Letter, H. P., McAdams, C. R., Puthoff, G. D., Smith, K. L., Steenburg, S. D., Banerjee, I., & Trivedi, H. (2022). Multireader evaluation of radiologist performance for COVID-19 detection on emergency department chest radiographs. Clinical Imaging, 82, 77–82. https://doi.org/10.1016/j.clinimag.2021.10.0181873-4499https://hdl.handle.net/1805/27870This article is made available for unrestricted research re-use and secondary analysis in any form or be any means with acknowledgement of the original source. These permissions are granted for the duration of the World Health Organization (WHO) declaration of COVID-19 as a global pandemic.BACKGROUND: Chest radiographs (CXR) are frequently used as a screening tool for patients with suspected COVID-19 infection pending reverse transcriptase polymerase chain reaction (RT-PCR) results, despite recommendations against this. We evaluated radiologist performance for COVID-19 diagnosis on CXR at the time of patient presentation in the Emergency Department (ED). MATERIALS AND METHODS: We extracted RT-PCR results, clinical history, and CXRs of all patients from a single institution between March and June 2020. 984 RT-PCR positive and 1043 RT-PCR negative radiographs were reviewed by 10 emergency radiologists from 4 academic centers. 100 cases were read by all radiologists and 1927 cases by 2 radiologists. Each radiologist chose the single best label per case: Normal, COVID-19, Other - Infectious, Other - Noninfectious, Non-diagnostic, and Endotracheal Tube. Cases labeled with endotracheal tube (246) or non-diagnostic (54) were excluded. Remaining cases were analyzed for label distribution, clinical history, and inter-reader agreement. RESULTS: 1727 radiographs (732 RT-PCR positive, 995 RT-PCR negative) were included from 1594 patients (51.2% male, 48.8% female, age 59 ± 19 years). For 89 cases read by all readers, there was poor agreement for RT-PCR positive (Fleiss Score 0.36) and negative (Fleiss Score 0.46) exams. Agreement between two readers on 1638 cases was 54.2% (373/688) for RT-PCR positive cases and 71.4% (679/950) for negative cases. Agreement was highest for RT-PCR negative cases labeled as Normal (50.4%, n = 479). Reader performance did not improve with clinical history or time between CXR and RT-PCR result. CONCLUSION: At the time of presentation to the emergency department, emergency radiologist performance is non-specific for diagnosing COVID-19.en-USPublic Health EmergencyChest X-rayCoronavirusCXRArticle