Korean Journal of Radiology

The Korean Society of Radiology (대한영상의학회)
권호 표지
DOI QR코드

DOI QR Code

Implementation of a Deep Learning-Based Computer-Aided Detection System for the Interpretation of Chest Radiographs in Patients Suspected for COVID-19

  • Eui Jin Hwang (Department of Radiology, Seoul National University College of Medicine) ;
  • Hyungjin Kim (Department of Radiology, Seoul National University College of Medicine) ;
  • Soon Ho Yoon (Department of Radiology, Seoul National University College of Medicine) ;
  • Jin Mo Goo (Department of Radiology, Seoul National University College of Medicine) ;
  • Chang Min Park (Department of Radiology, Seoul National University College of Medicine)
  • Received : 2020.04.27
  • Accepted : 2020.05.28
  • Published : 2020.10.01
Download PDF
⟨ Previous Next ⟩

Abstract

Objective: To describe the experience of implementing a deep learning-based computer-aided detection (CAD) system for the interpretation of chest X-ray radiographs (CXR) of suspected coronavirus disease (COVID-19) patients and investigate the diagnostic performance of CXR interpretation with CAD assistance. Materials and Methods: In this single-center retrospective study, initial CXR of patients with suspected or confirmed COVID-19 were investigated. A commercialized deep learning-based CAD system that can identify various abnormalities on CXR was implemented for the interpretation of CXR in daily practice. The diagnostic performance of radiologists with CAD assistance were evaluated based on two different reference standards: 1) real-time reverse transcriptase-polymerase chain reaction (rRT-PCR) results for COVID-19 and 2) pulmonary abnormality suggesting pneumonia on chest CT. The turnaround times (TATs) of radiology reports for CXR and rRT-PCR results were also evaluated. Results: Among 332 patients (male:female, 173:159; mean age, 57 years) with available rRT-PCR results, 16 patients (4.8%) were diagnosed with COVID-19. Using CXR, radiologists with CAD assistance identified rRT-PCR positive COVID-19 patients with sensitivity and specificity of 68.8% and 66.7%, respectively. Among 119 patients (male:female, 75:44; mean age, 69 years) with available chest CTs, radiologists assisted by CAD reported pneumonia on CXR with a sensitivity of 81.5% and a specificity of 72.3%. The TATs of CXR reports were significantly shorter than those of rRT-PCR results (median 51 vs. 507 minutes; p < 0.001). Conclusion: Radiologists with CAD assistance could identify patients with rRT-PCR-positive COVID-19 or pneumonia on CXR with a reasonably acceptable performance. In patients suspected with COVID-19, CXR had much faster TATs than rRT-PCRs.

Keywords

Acknowledgement

The present study was supported by a grant (grant number: 03-2019-0190) from the Seoul National University Hospital research fund.

References

  1. Zhu N, Zhang D, Wang W, Li X, Yang B, Song J, et al. A novel coronavirus from patients with pneumonia in China, 2019. N Engl J Med 2020;382:727-733
  2. National Authorities. Coronavirus disease (COVID-19). Situation report-127. World Health Organization, 2020. Available at: https://www.who.int/docs/default-source/coronaviruse/situation-reports/20200526-covid-19-sitrep-127.pdf?sfvrsn=7b6655ab_8. Accessed May 26, 2020
  3. Interim guidelines for collecting, handling, and testing clinical specimens from persons for coronavirus disease 2019 (COVID-19). Centers for Disease Control and Prevention Web site. https://www.cdc.gov/coronavirus/2019-nCoV/lab/guidelines-clinical-specimens.html. Published May 22, 2020. Accessed May 26, 2020
  4. Ai T, Yang Z, Hou H, Zhan C, Chen C, Lv W, et al. Correlation of chest CT and RT-PCR testing in coronavirus disease 2019 (COVID-19) in China: a report of 1014 cases. Radiology 2020 Feb 26 [Epub]. https://doi.org/10.1148/radiol.2020200642
  5. Fang Y, Zhang H, Xie J, Lin M, Ying L, Pang P, et al. Sensitivity of chest CT for COVID-19: comparison to RT-PCR. Radiology 2020 Feb 19 [Epub]. https://doi.org/10.1148/radiol.2020200432
  6. Xie X, Zhong Z, Zhao W, Zheng C, Wang F, Liu J. Chest CT for typical 2019-nCoV pneumonia: relationship to negative RT-PCR testing. Radiology 2020 Feb 12 [Epub]. https://doi.org/10.1148/radiol.2020200343
  7. ACR recommendations for the use of chest radiography and computed tomography (CT) for suspected COVID-19 infection. American College of Radiology Web site. https://www.acr.org/Advocacy-and-Economics/ACR-Position-Statements/Recommendations-for-Chest-Radiography-and-CT-forSuspected-COVID19-Infection. Published March 11, 2020. Accessed May 26, 2020
  8. STR/ASER COVID-19 position statement. Society of Thoracic Radiology Web site. https://thoracicrad.org/?page_id=2879. Published March 11, 2020. Accessed May 26, 2020
  9. Expert Panel on Thoracic Imaging; Jokerst C, Chung JH, Ackman JB, Carter B, Colletti PM, Crabtree TD, et al. ACR appropriateness Criteria® acute respiratory illness in immunocompetent patients. J Am Coll Radiol 2018;15:S240-S251
  10. Mandell LA, Wunderink RG, Anzueto A, Bartlett JG, Campbell GD, Dean NC, et al. Infectious Diseases Society of America/American Thoracic Society consensus guidelines on the management of community-acquired pneumonia in adults. Clin Infect Dis 2007;44 Suppl 2:S27-S72
  11. Esayag Y, Nikitin I, Bar-Ziv J, Cytter R, Hadas-Halpern I, Zalut T, et al. Diagnostic value of chest radiographs in bedridden patients suspected of having pneumonia. Am J Med 2010;123:88.e1-e5
  12. Self WH, Courtney DM, McNaughton CD, Wunderink RG, Kline JA. High discordance of chest X-ray and computed tomography for detection of pulmonary opacities in ED patients: implications for diagnosing pneumonia. Am J Emerg Med 2013;31:401-405
  13. Donald JJ, Barnard SA. Common patterns in 558 diagnostic radiology errors. J Med Imaging Radiat Oncol 2012;56:173-178
  14. Chumbita M, Cilloniz C, Puerta-Alcalde P, Moreno-Garcia E, Sanjuan G, Garcia-Pouton N, et al. Can artificial intelligence improve the management of pneumonia. J Clin Med 2020;9:248
  15. Hwang EJ, Park S, Jin KN, Kim JI, Choi SY, Lee JH, et al. Development and validation of a deep learning-based automated detection algorithm for major thoracic diseases on chest radiographs. JAMA Netw Open 2019;2:e191095
  16. Leisenring W, Alonzo T, Pepe MS. Comparisons of predictive values of binary medical diagnostic tests for paired designs. Biometrics 2000;56:345-351
  17. Coronavirus disease-19, Republic of Korea. Latest updates, cases in Korea. Korean Ministry of Health and Welfare Web site. http://ncov.mohw.go.kr/bdBoardList_Real.do?brdId=1&brdGubun=11&ncvContSeq=&contSeq=&board_id=&gubun=. Published April 4, 2020. Accessed April 4, 2020
  18. Korean Society of Infectious Diseases, Korean Society of Pediatric Infectious Diseases, Korean Society of Epidemiology, Korean Society for Antimicrobial Therapy, Korean Society for Healthcare-associated Infection Control and Prevention, Korea Centers for Disease Control and Prevention, et al. Report on the epidemiological features of coronavirus disease 2019 (COVID-19) outbreak in the Republic of Korea from January 19 to March 2, 2020. J Korean Med Sci 2020;35:e112
  19. Yoon SH, Lee KH, Kim JY, Lee YK, Ko H, Kim KH, et al. Chest radiographic and CT findings of the 2019 novel coronavirus disease (COVID-19): analysis of nine patients treated in Korea. Korean J Radiol 2020;21:494-500
  20. Ng MY, Lee EY, Yang J, Yang F, Li X, Wang H, et al. Imaging profile of the COVID-19 infection: radiologic findings and literature review. Radiology: Cardiothoracic Imaging 2020;2:e200034
  21. Wong HYF, Lam HYS, Fong AH, Leung ST, Chin TW, Lo CSY, et al. Frequency and distribution of chest radiographic findings in COVID-19 positive patients. Radiology 2020 Mar 27 [Epub]. https://doi.org/10.1148/radiol.2020201160
  22. Chung M, Bernheim A, Mei X, Zhang N, Huang M, Zeng X, et al. CT imaging features of 2019 novel coronavirus (2019-nCoV). Radiology 2020;295:202-207
  23. Shi H, Han X, Jiang N, Cao Y, Alwalid O, Gu J, et al. Radiological findings from 81 patients with COVID-19 pneumonia in Wuhan, China: a descriptive study. Lancet Infect Dis 2020;20:425-434
  24. Zhao W, Zhong Z, Xie X, Yu Q, Liu J. Relation between chest CT findings and clinical conditions of coronavirus disease (COVID-19) pneumonia: a multicenter study. AJR Am J Roentgenol 2020;214:1072-1077
  25. Antonio GE, Ooi CG, Wong KT, Tsui EL, Wong JS, Sy AN, et al. Radiographic-clinical correlation in severe acute respiratory syndrome: study of 1373 patients in Hong Kong. Radiology 2005;237:1081-1090
  26. Hui DS, Wong KT, Antonio GE, Lee N, Wu A, Wong V, et al. Severe acute respiratory syndrome: correlation between clinical outcome and radiologic features. Radiology 2004;233:579-585
  27. Ko SF, Lee TY, Huang CC, Cheng YF, Ng SH, Kuo YL, et al. Severe acute respiratory syndrome: prognostic implications of chest radiographic findings in 52 patients. Radiology 2004;233:173-181
  28. Hwang EJ, Nam JG, Lim WH, Park SJ, Jeong YS, Kang JH, et al. Deep learning for chest radiograph diagnosis in the emergency department. Radiology 2019;293:573-580
  29. Orsi MA, Oliva AG, Cellina M. Radiology department preparedness for COVID-19: facing an unexpected outbreak of the disease. Radiology 2020;295:E8
  30. Hwang EJ, Park CM. Clinical implementation of deep learning in thoracic radiology: potential applications and challenges. Korean J Radiol 2020;21:511-525