Using ImageBERT to improve performance of multi-class Chest Xray classification

Abstract

Pulmonary edema is a medical condition that is often related to life-threatening heart-related complications. Several recent studies have demonstrated that machine learning models using deep learning (DL) methods are able to identify anomalies on chest X-rays (CXR) as well as trained radiologists. Yet, there are limited/no studies that have integrated these models in clinical radiology workflows. The objective of this project is to identify state-of-the-art DL algorithms and integrate the classification results into the radiology workflow, more specifically in a DICOM Viewer, so that radiologists can use it as a clinical decision support. Our proof-of-concept (POC) is to detect the presence/absence of edema in chest radiographs obtained from the CheXpert dataset. We implemented the state-of-the-art deep learning methods for image classification -ResNet50, VGG16 and Inception v4 using the FastAI library and PyTorch on 77,408 CXR which have classified the presence/absence of edema in the images with an accuracy of 65%, 70% and 65% respectively on a test dataset of about 202 images. The CXR were converted to DICOM format using the img2dcm utility of DICOM ToolKit (DCMTK), and later uploaded to the Orthanc PACS, which was connected to the OHIF Viewer. This is the first study that has integrated the machine learning outcomes into the clinical workflow in order to improve the decision-making process by implementing object detection and instance segmentation algorithms.