Korean Journal of Radiology

  • 제22권4호
  • /
  • Pages.612-623
  • /
  • 2021
  • /
  • 1229-6929(pISSN)
  • /
  • 2005-8330(eISSN)
대한영상의학회 (The Korean Society of Radiology)
권호 표지
DOI QR코드

DOI QR Code

Diagnostic Performance of a New Convolutional Neural Network Algorithm for Detecting Developmental Dysplasia of the Hip on Anteroposterior Radiographs

  • Hyoung Suk Park (Division of Medical Mathematics, National Institute for Mathematical Sciences) ;
  • Kiwan Jeon (Division of Medical Mathematics, National Institute for Mathematical Sciences) ;
  • Yeon Jin Cho (Department of Radiology, Seoul National University Hospital) ;
  • Se Woo Kim (Department of Radiology, Seoul National University Hospital) ;
  • Seul Bi Lee (Department of Radiology, Seoul National University Hospital) ;
  • Gayoung Choi (Department of Radiology, Seoul National University Hospital) ;
  • Seunghyun Lee (Department of Radiology, Seoul National University Hospital) ;
  • Young Hun Choi (Department of Radiology, Seoul National University Hospital) ;
  • Jung-Eun Cheon (Department of Radiology, Seoul National University Hospital) ;
  • Woo Sun Kim (Department of Radiology, Seoul National University Hospital) ;
  • Young Jin Ryu (Department of Radiology, Seoul National University Bundang Hospital) ;
  • Jae-Yeon Hwang (Department of Radiology, Pusan National University Yangsan Hospital)
  • 투고 : 2020.01.21
  • 심사 : 2020.07.22
  • 발행 : 2021.04.01
PDF 다운로드
⟨ 이전 논문 다음 논문 ⟩

초록

Objective: To evaluate the diagnostic performance of a deep learning algorithm for the automated detection of developmental dysplasia of the hip (DDH) on anteroposterior (AP) radiographs. Materials and Methods: Of 2601 hip AP radiographs, 5076 cropped unilateral hip joint images were used to construct a dataset that was further divided into training (80%), validation (10%), or test sets (10%). Three radiologists were asked to label the hip images as normal or DDH. To investigate the diagnostic performance of the deep learning algorithm, we calculated the receiver operating characteristics (ROC), precision-recall curve (PRC) plots, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) and compared them with the performance of radiologists with different levels of experience. Results: The area under the ROC plot generated by the deep learning algorithm and radiologists was 0.988 and 0.988-0.919, respectively. The area under the PRC plot generated by the deep learning algorithm and radiologists was 0.973 and 0.618-0.958, respectively. The sensitivity, specificity, PPV, and NPV of the proposed deep learning algorithm were 98.0, 98.1, 84.5, and 99.8%, respectively. There was no significant difference in the diagnosis of DDH by the algorithm and the radiologist with experience in pediatric radiology (p = 0.180). However, the proposed model showed higher sensitivity, specificity, and PPV, compared to the radiologist without experience in pediatric radiology (p < 0.001). Conclusion: The proposed deep learning algorithm provided an accurate diagnosis of DDH on hip radiographs, which was comparable to the diagnosis by an experienced radiologist.

키워드

과제정보

This research was supported by Research Program 2019 funded by Seoul National University College of Medicine Research Foundation. H.S.P and K.J. were supported by the National Institute for Mathematical Sciences (NIMS) grant funded by the Korean government (No. NIMS-B20900000). This work utilized a software for screening of DDH (C-2019-015787) developed by National Institute for Mathematical Sciences.

참고문헌

  1. Siegel MJ. Pediatric sonography, 4th ed. Philadelphia: Lippincott Williams & Wilkins, 2011:607
  2. Kotlarsky P, Haber R, Bialik V, Eidelman M. Developmental dysplasia of the hip: what has changed in the last 20 years? World J Orthop 2015;6:886-901 https://doi.org/10.5312/wjo.v6.i11.886
  3. Starr V, Ha BY. Imaging update on developmental dysplasia of the hip with the role of MRI. AJR Am J Roentgenol 2014;203:1324-1335 https://doi.org/10.2214/AJR.13.12449
  4. Price KR, Dove R, Hunter JB. The use of X-ray at 5 months in a selective screening programme for developmental dysplasia of the hip. J Child Orthop 2011;5:195-200 https://doi.org/10.1007/s11832-011-0338-2
  5. Coley, BD. Caffey's pediatric diagnostic imaging, 13th ed. Philadelphia: Elsevier, 2019:1316-1322
  6. Sahin S, Akata E, Sahin O, Tuncay C, Ozkan H. A novel computer-based method for measuring the acetabular angle on hip radiographs. Acta Orthop Traumatol Turc 2017;51:155-159 https://doi.org/10.1016/j.aott.2016.09.002
  7. Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, et al. Imagenet large scale visual recognition challenge. Int J Comput Vis 2015;115:211-252 https://doi.org/10.1007/s11263-015-0816-y
  8. Kim JR, Shim WH, Yoon HM, Hong SH, Lee JS, Cho YA, et al. Computerized bone age estimation using deep learning based program: evaluation of the accuracy and efficiency. AJR Am J Roentgenol 2017;209:1374-1380 https://doi.org/10.2214/AJR.17.18224
  9. Cicero M, Bilbily A, Colak E, Dowdell T, Gray B, Perampaladas K, et al. Training and validating a deep convolutional neural network for computer-aided detection and classification of abnormalities on frontal chest radiographs. Invest Radiol 2017;52:281-287 https://doi.org/10.1097/RLI.0000000000000341
  10. Mannil M, von Spiczak J, Manka R, Alkadhi H. Texture analysis and machine learning for detecting myocardial infarction in noncontrast low-dose computed tomography: unveiling the invisible. Invest Radiol 2018;53:338-343 https://doi.org/10.1097/RLI.0000000000000448
  11. Ciritsis A, Rossi C, Wurnig MC, Phi Van V, Boss A. Intravoxel incoherent motion: model-free determination of tissue type in abdominal organs using machine learning. Invest Radiol 2017;52:747-757 https://doi.org/10.1097/RLI.0000000000000400
  12. Becker AS, Marcon M, Ghafoor S, Wurnig MC, Frauenfelder T, Boss A. Deep learning in mammography: diagnostic accuracy of a multipurpose image analysis software in the detection of breast cancer. Invest Radiol 2017;52:434-440 https://doi.org/10.1097/RLI.0000000000000358
  13. Kim Y, Lee KJ, Sunwoo L, Choi D, Nam CM, Cho J, et al. Deep learning in diagnosis of maxillary sinusitis using conventional radiography. Invest Radiol 2019;54:7-15 https://doi.org/10.1097/RLI.0000000000000503
  14. Rayan JC, Reddy N, Kan JH, Zhang W, Annapragada A. Binomial classification of pediatric elbow fractures using a deep learning multiview approach emulating radiologist decision making. Radiology: Artificial Intelligence 2019;1:e180015
  15. Zonoobi D, Hareendranathan A, Mostofi E, Mabee M, Pasha S, Cobzas D, et al. Developmental hip dysplasia diagnosis at three-dimensional US: a multicenter study. Radiology 2018;287:1003-1015 https://doi.org/10.1148/radiol.2018172592
  16. Hareendranathan AR, Zonoobi D, Mabee M, Cobzas D, Punithakumar K, Noga M, et al. Toward automatic diagnosis of hip dysplasia from 2D ultrasound. IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017); 2017 Apr 18-21; Melbourne, Australia: IEEE; p. 982-985
  17. Li Q, Zhong L, Huang H, Liu H, Qin Y, Wang Y, et al. Auxiliary diagnosis of developmental dysplasia of the hip by automated detection of Sharp's angle on standardized anteroposterior pelvic radiographs. Medicine (Baltimore) 2019;98:e18500
  18. Krizhevsky A, Sutskever I, Hinton GE. Advances in neural information processing systems. In: Krizhevsky A, Sutskever I, Hinton GE, eds. Imagenet classification with deep convolutional neural networks. San Diego: Neural Information Processing Systems Foundation, 2012:1097-1105
  19. Nair V, Hinton GE. Rectified linear units improve restricted boltzmann machines. Proceedings of the 27th International Conference on Machine Learning (ICML-10); 2010 Jun 21-24; Haifa, Israel; ICML; p. 807-814
  20. Ioffe S, Szegedy C. Batch normalization: accelerating deep network training by reducing internal covariate shift. arXiv preprint 2015;arXiv:1502.03167
  21. Bishop CM. Pattern recognition and machine learning. New York: Springer, 2006:203-204
  22. Goodfellow I, Bengio Y, Courville A, Bengio Y. Deep learning. Cambridge: MIT Press, 2016:131-132
  23. Kingma DP, Ba J. Adam: a method for stochastic optimization. arXiv preprint 2014;arXiv:1412.6980
  24. Abadi M, Agarwal A, Barham P, Brevdo E, Chen Z, Citro C, et al. Tensorflow: large-scale machine learning on heterogeneous distributed systems. arXiv preprint 2016;arXiv:1603.04467
  25. Selvaraju RR, Cogswell M, Das A, Vedantam R, Parikh D, Batra D. Grad-cam: visual explanations from deep networks via gradient-based localization. Proceedings of the IEEE International Conference on Computer Vision; 2017 Oct 22-29; Venice, Italy: ICCV; p. 618-626
  26. Saito T, Rehmsmeier M. The precision-recall plot is more informative than the ROC plot when evaluating binary classifiers on imbalanced datasets. PLoS One 2015;10:e0118432
  27. Shaw BA, Segal LS; Section on Orthopaedics. Evaluation and referral for developmental dysplasia of the hip in infants. Pediatrics 2016;138:e20163107
  28. Shipman SA, Helfand M, Moyer VA, Yawn BP. Screening for developmental dysplasia of the hip: a systematic literature review for the US preventive services task force. Pediatrics 2006;117:e557-e576 https://doi.org/10.1542/peds.2005-1597
  29. Holen KJ, Tegnander A, Bredland T, Johansen OJ, Saether OD, Eik-Nes SH, et al. Universal or selective screening of the neonatal hip using ultrasound? A prospective, randomised trial of 15,529 newborn infants. J Bone Joint Surg Br 2002;84:886-890 https://doi.org/10.1302/0301-620X.84B6.0840886
  30. Woolacott NF, Puhan MA, Steurer J, Kleijnen J. Ultrasonography in screening for developmental dysplasia of the hip in newborns: systematic review. BMJ 2005;330:1413
  31. Karmazyn BK, Gunderman RB, Coley BD, Blatt ER, Bulas D, Fordham L, et al. ACR appropriateness criteria on developmental dysplasia of the hip--child. J Am Coll Radiol 2009;6:551-557 https://doi.org/10.1016/j.jacr.2009.04.008
  32. Peterlein CD, Schuttler KF, Lakemeier S, Timmesfeld N, Gorg C, Fuchs-Winkelmann S, et al. Reproducibility of different screening classifications in ultrasonography of the newborn hip. BMC Pediatr 2010;10:98
  33. Hell AK, Becker JC, Ruhmann O, Lewinski Gv, Lazovic D. Inter- und intraindividuelle Messabweichungen in der Sauglingshuftsonografie nach Graf. Z Orthop Unfall 2008;146:624-629 https://doi.org/10.1055/s-2008-1038477
  34. Quader N, Schaeffer EK, Hodgson AJ, Abugharbieh R, Mulpuri K. A systematic review and meta-analysis on the reproducibility of ultrasound-based metrics for assessing developmental dysplasia of the hip. J Pediatr Orthop 2018;38:e305-e311 https://doi.org/10.1097/BPO.0000000000001179
  35. Tudor A, Sestan B, Rakovac I, Schnurrer Luke-Vrbanic' T, Prpic' T, Rubinic' D, et al. The rational strategies for detecting developmental dysplasia of the hip at the age of 4-6 months old infants: a prospective study. Coll Antropol 2007;31:475-481
  36. Garvey M, Donoghue VB, Gorman WA, O'Brien N, Murphy JF. Radiographic screening at four months of infants at risk for congenital hip dislocation. J Bone Joint Surg Br 1992;74:704-707 https://doi.org/10.1302/0301-620X.74B5.1527117
  37. Nguyen JC, Dorfman SR, Rigsby CK, Iyer RS, Alazraki AL, Anupindi SA, et al. ACR appropriateness criteria® developmental dysplasia of the hip-child. J Am Coll Radiol 2019;16:S94-S103 https://doi.org/10.1016/j.jacr.2019.02.014