Interrater Reliability of NI-RADS on Posttreatment PET/Contrast-enhanced CT Scans in Head and Neck Squamous Cell Carcinoma

Abstract

Purpose: To evaluate the interrater reliability among radiologists examining posttreatment head and neck squamous cell carcinoma (HNSCC) fluorodeoxyglucose PET/contrast-enhanced CT (CECT) scans using Neck Imaging Reporting and Data System (NI-RADS). Materials and Methods: In this retrospective study, images in 80 patients with HNSCC who underwent posttreatment surveillance PET/CECT and immediate prior comparison CECT or PET/CECT (from June 2014 to July 2016) were uploaded to the American College of Radiology's cloud-based website, Cortex. Eight radiologists from seven institutions with variable NI-RADS experience independently evaluated each case and assigned an appropriate prose description and NI-RADS category for the primary site and the neck site. Five of these individuals were experienced readers (> 5 years of experience), and three were novices (< 5 years of experience). In total, 640 lexicon-based and NI-RADS categories were assigned to lesions among the 80 included patients by the eight radiologists. Light generalization of Cohen κ for interrater reliability was performed. Results: Of the 80 included patients (mean age, 63 years ± 10 [standard deviation]), there were 58 men (73%); 60 patients had stage IV HNSCC (75%), and the most common tumor location was oropharynx (n = 32; 40%). Light κ for lexicon was 0.30 (95% CI: 0.23, 0.36) at the primary site and 0.31 (95% CI: 0.24, 0.37) at the neck site. Light κ for NI-RADS category was 0.55 (95% CI: 0.46, 0.63) at the primary site and 0.60 (95% CI: 0.48, 0.69) at the neck site. Percent agreement between lexicon and correlative NI-RADS category was 84.4% (540 of 640) at the primary site and 92.6% (593 of 640) at the neck site. There was no significant difference in interobserver agreement among the experienced versus novice raters. Conclusion: Moderate agreement was achieved among eight radiologists using NI-RADS at posttreatment HNSCC surveillance imaging.