Korean Journal of Radiology

The Korean Society of Radiology (대한영상의학회)
권호 표지
DOI QR코드

DOI QR Code

How to Determine If One Diagnostic Method, Such as an Artificial Intelligence Model, is Superior to Another: Beyond Performance Metrics

  • Seong Ho Park (Department of Radiology and Research Institute of Radiology, Asan Medical Center, University of Ulsan College of Medicine) ;
  • Ah-Ram Sul (Division of Healthcare Research Outcomes Research, National Evidence-based Healthcare Collaborating Agency) ;
  • Kyunghwa Han (Department of Radiology, Research Institute of Radiological Science and Center for Clinical Imaging Data Science, Yonsei University College of Medicine) ;
  • Yu Sub Sung (Clinical Research Center, Asan Medical Center)
  • Received : 2023.05.14
  • Accepted : 2023.06.09
  • Published : 2023.07.01
Download PDF
⟨ Previous Next ⟩

Abstract

Keywords

Acknowledgement

This study was supported by the National Evidencebased Healthcare Collaborating Agency in South Korea (NECA-A-23-002).

References

  1. Park SH, Han K, Jang HY, Park JE, Lee JG, Kim DW, et al. Methods for clinical evaluation of artificial intelligence algorithms for medical diagnosis. Radiology 2023;306:20-31 https://doi.org/10.1148/radiol.220182
  2. Hwang EJ, Goo JM, Yoon SH, Beck KS, Seo JB, Choi BW, et al. Use of artificial intelligence-based software as medical devices for chest radiography: a position paper from the Korean Society of Thoracic Radiology. Korean J Radiol 2021;22:1743-1748 https://doi.org/10.3348/kjr.2021.0544
  3. Hwang EJ, Park S, Jin KN, Kim JI, Choi SY, Lee JH, et al. Development and validation of a deep learning-based automated detection algorithm for major thoracic diseases on chest radiographs. JAMA Netw Open 2019;2:e191095
  4. Vickers AJ, Elkin EB. Decision curve analysis: a novel method for evaluating prediction models. Med Decis Making 2006;26:565-574 https://doi.org/10.1177/0272989X06295361
  5. Habibzadeh F, Habibzadeh P, Yadollahie M. On determining the most appropriate test cut-off value: the case of tests with continuous results. Biochem Med (Zagreb) 2016;26:297-307 https://doi.org/10.11613/BM.2016.034
  6. Obuchowski NA. Receiver operating characteristic curves and their use in radiology. Radiology 2003;229:3-8 https://doi.org/10.1148/radiol.2291010898
  7. Vickers AJ, Cronin AM, Gonen M. A simple decision analytic solution to the comparison of two binary diagnostic tests. Stat Med 2013;32:1865-1876 https://doi.org/10.1002/sim.5601
  8. Park SH, Choi JI, Fournier L, Vasey B. Randomized clinical trials of artificial intelligence in medicine: why, when, and how? Korean J Radiol 2022;23:1119-1125 https://doi.org/10.3348/kjr.2022.0834
  9. Liu X, Rivera SC, Moher D, Calvert MJ, Denniston AK; SPIRITAI and CONSORT-AI Working Group. Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: the CONSORT-AI Extension. BMJ 2020;370:m3164
  10. Vasey B, Nagendran M, Campbell B, Clifton DA, Collins GS, Denaxas S, et al. Reporting guideline for the early stage clinical evaluation of decision support systems driven by artificial intelligence: DECIDE-AI. BMJ 2022;377:e070904