Clinical Endoscopy

The Korean Society of Gastrointestinal Endoscopy (대한소화기내시경학회)
권호 표지
DOI QR코드

DOI QR Code

Preparation of image databases for artificial intelligence algorithm development in gastrointestinal endoscopy

  • Chang Bong Yang (Department of Internal Medicine, Dongguk University Ilsan Hospital, Dongguk University College of Medicine) ;
  • Sang Hoon Kim (Department of Internal Medicine, Dongguk University Ilsan Hospital, Dongguk University College of Medicine) ;
  • Yun Jeong Lim (Department of Internal Medicine, Dongguk University Ilsan Hospital, Dongguk University College of Medicine)
  • Received : 2021.09.10
  • Accepted : 2022.03.07
  • Published : 2022.09.30
Download PDF
⟨ Previous Next ⟩

Abstract

Over the past decade, technological advances in deep learning have led to the introduction of artificial intelligence (AI) in medical imaging. The most commonly used structure in image recognition is the convolutional neural network, which mimics the action of the human visual cortex. The applications of AI in gastrointestinal endoscopy are diverse. Computer-aided diagnosis has achieved remarkable outcomes with recent improvements in machine-learning techniques and advances in computer performance. Despite some hurdles, the implementation of AI-assisted clinical practice is expected to aid endoscopists in real-time decision-making. In this summary, we reviewed state-of-the-art AI in the field of gastrointestinal endoscopy and offered a practical guide for building a learning image dataset for algorithm development.

Keywords

Acknowledgement

This research was supported by a grant from the Korean Health Technology R&D Project through the Korean Health Industry Development Institute (KHIDI), funded by the Ministry of Health and Welfare, Republic of Korea (No: HI19C0665).

References

  1. Ruffle JK, Farmer AD, Aziz Q. Artificial intelligence-assisted gastroenterology- promises and pitfalls. Am J Gastroenterol 2019;114:422-428. https://doi.org/10.1038/s41395-018-0268-4
  2. Samuel AL. Some studies in machine learning using the game of checkers. IBM J Res Dev 1959;3:210-229. https://doi.org/10.1147/rd.33.0210
  3. Rajkomar A, Dean J, Kohane I. Machine learning in medicine. N Engl J Med 2019;380:1347-1358. https://doi.org/10.1056/NEJMra1814259
  4. McCulloch WS, Pitts W. A logical calculus of the ideas immanent in nervous activity. 1943. Bull Math Biol 1990;52:99-115. https://doi.org/10.1016/S0092-8240(05)80006-0
  5. Litjens G, Ciompi F, Wolterink JM, et al. State-of-the-art deep learning in cardiovascular image analysis. JACC Cardiovasc Imaging 2019;12:1549-1565. https://doi.org/10.1016/j.jcmg.2019.06.009
  6. Hinton GE, Salakhutdinov RR. Reducing the dimensionality of data with neural networks. Science 2006;313:504-507. https://doi.org/10.1126/science.1127647
  7. Le Cun Y, Boser B, Denker JS, et al. Handwritten digit recognition with a back-propagation network. In: Touretzky DS, editor. Advances in neural information processing systems 2. San Francisco (CA): Morgan Kaufmann Publishers Inc.; 1990. p. 396-404.
  8. Ahmad OF, Soares AS, Mazomenos E, et al. Artificial intelligence and computer-aided diagnosis in colonoscopy: current evidence and future directions. Lancet Gastroenterol Hepatol 2019;4:71-80. https://doi.org/10.1016/S2468-1253(18)30282-6
  9. Krizhevsky A, Sutskever I, Hinton GE. ImageNet classification with deep convolutional neural networks. Commun ACM 2017;60:84-90. https://doi.org/10.1145/3065386
  10. He K, Zhang X, Ren S, et al. Deep residual learning for image recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR); 2016 Jun 27-30; Las Vegas, NV. p. 770-778.
  11. Litjens G, Kooi T, Bejnordi BE, et al. A survey on deep learning in medical image analysis. Med Image Anal 2017;42:60-88. https://doi.org/10.1016/j.media.2017.07.005
  12. Topol EJ. High-performance medicine: the convergence of human and artificial intelligence. Nat Med 2019;25:44-56. https://doi.org/10.1038/s41591-018-0300-7
  13. Billah M, Waheed S, Rahman MM. An automatic gastrointestinal polyp detection system in video endoscopy using fusion of color wavelet and convolutional neural network features. Int J Biomed Imaging 2017;2017:9545920.
  14. Zhang R, Zheng Y, Mak TW, et al. Automatic detection and classification of colorectal polyps by transferring low-level CNN features from nonmedical domain. IEEE J Biomed Health Inform 2017;21:41-47. https://doi.org/10.1109/JBHI.2016.2635662
  15. Byrne MF, Chapados N, Soudan F, et al. Real-time differentiation of adenomatous and hyperplastic diminutive colorectal polyps during analysis of unaltered videos of standard colonoscopy using a deep learning model. Gut 2019;68:94-100. https://doi.org/10.1136/gutjnl-2017-314547
  16. Repici A, Badalamenti M, Maselli R, et al. Efficacy of real-time computer-aided detection of colorectal neoplasia in a randomized trial. Gastroenterology 2020;159:512-520.e7. https://doi.org/10.1053/j.gastro.2020.04.062
  17. Komeda Y, Handa H, Matsui R, et al. Artificial intelligence-based endoscopic diagnosis of colorectal polyps using residual networks. PLoS One 2021;16:e0253585.
  18. Misawa M, Kudo SE, Mori Y, et al. Artificial intelligence-assisted polyp detection for colonoscopy: initial experience. Gastroenterology 2018;154:2027-2029.e3. https://doi.org/10.1053/j.gastro.2018.04.003
  19. Wang P, Xiao X, Glissen Brown JR, et al. Development and validation of a deep-learning algorithm for the detection of polyps during colonoscopy. Nat Biomed Eng 2018;2:741-748. https://doi.org/10.1038/s41551-018-0301-3
  20. Itoh T, Kawahira H, Nakashima H, et al. Deep learning analyzes Helicobacter pylori infection by upper gastrointestinal endoscopy images. Endosc Int Open 2018;6:E139-E144. https://doi.org/10.1055/s-0043-120830
  21. Shichijo S, Nomura S, Aoyama K, et al. Application of convolutional neural networks in the diagnosis of Helicobacter pylori infection based on endoscopic images. EBioMedicine 2017;25:106-111. https://doi.org/10.1016/j.ebiom.2017.10.014
  22. Kudo SE, Misawa M, Mori Y, et al. Artificial intelligence-assisted system improves endoscopic identification of colorectal neoplasms. Clin Gastroenterol Hepatol 2020;18:1874-1881.e2. https://doi.org/10.1016/j.cgh.2019.09.009
  23. van der Sommen F, Zinger S, Curvers WL, et al. Computer-aided detection of early neoplastic lesions in Barrett's esophagus. Endoscopy 2016;48:617-624. https://doi.org/10.1055/s-0042-105284
  24. Horie Y, Yoshio T, Aoyama K, et al. Diagnostic outcomes of esophageal cancer by artificial intelligence using convolutional neural networks. Gastrointest Endosc 2019;89:25-32. https://doi.org/10.1016/j.gie.2018.07.037
  25. Kanesaka T, Lee TC, Uedo N, et al. Computer-aided diagnosis for identifying and delineating early gastric cancers in magnifying narrow-band imaging. Gastrointest Endosc 2018;87:1339-1344. https://doi.org/10.1016/j.gie.2017.11.029
  26. Wu L, Zhou W, Wan X, et al. A deep neural network improves endoscopic detection of early gastric cancer without blind spots. Endoscopy 2019;51:522-531. https://doi.org/10.1055/a-0855-3532
  27. Zhu Y, Wang QC, Xu MD, et al. Application of convolutional neural network in the diagnosis of the invasion depth of gastric cancer based on conventional endoscopy. Gastrointest Endosc 2019;89:806-815. https://doi.org/10.1016/j.gie.2018.11.011
  28. Ding Z, Shi H, Zhang H, et al. Gastroenterologist-level identification of small-bowel diseases and normal variants by capsule endoscopy using a deep-learning model. Gastroenterology 2019;157:1044-1054. https://doi.org/10.1053/j.gastro.2019.06.025
  29. Li P, Li Z, Gao F, et al. Convolutional neural networks for intestinal hemorrhage detection in wireless capsule endoscopy images. 2017 IEEE International Conference on Multimedia and Expo (ICME); 2017 Jul 10-14; Hong Kong, China. p. 1518-1523.
  30. Aoki T, Yamada A, Aoyama K, et al. Automatic detection of erosions and ulcerations in wireless capsule endoscopy images based on a deep convolutional neural network. Gastrointest Endosc 2019;89:357-363.e2. https://doi.org/10.1016/j.gie.2018.10.027
  31. Klang E, Barash Y, Margalit RY, et al. Deep learning algorithms for automated detection of Crohn's disease ulcers by video capsule endoscopy. Gastrointest Endosc 2020;91:606-613. https://doi.org/10.1016/j.gie.2019.11.012
  32. Corley DA, Levin TR, Doubeni CA. Adenoma detection rate and risk of colorectal cancer and death. N Engl J Med 2014;370:2539-2541. https://doi.org/10.1056/NEJMc1405329
  33. Urban G, Tripathi P, Alkayali T, et al. Deep learning localizes and identifies polyps in real time with 96% accuracy in screening colonoscopy. Gastroenterology 2018;155:1069-1078.e8. https://doi.org/10.1053/j.gastro.2018.06.037
  34. Gong EJ, Lee JH, Jung K, et al. Characteristics of missed simultaneous gastric lesions based on double-check analysis of the endoscopic image. Clin Endosc 2017;50:261-269. https://doi.org/10.5946/ce.2016.056
  35. Lui TK, Tsui VW, Leung WK. Accuracy of artificial intelligence-assisted detection of upper GI lesions: a systematic review and meta-analysis. Gastrointest Endosc 2020;92:821-830.e9. https://doi.org/10.1016/j.gie.2020.06.034
  36. Alsop BR, Sharma P. Esophageal cancer. Gastroenterol Clin North Am 2016;45:399-412. https://doi.org/10.1016/j.gtc.2016.04.001
  37. Kiesslich R, Burg J, Vieth M, et al. Confocal laser endoscopy for diagnosing intraepithelial neoplasias and colorectal cancer in vivo. Gastroenterology 2004;127:706-713. https://doi.org/10.1053/j.gastro.2004.06.050
  38. Mori Y, Kudo S, Ikehara N, et al. Comprehensive diagnostic ability of endocytoscopy compared with biopsy for colorectal neoplasms: a prospective randomized noninferiority trial. Endoscopy 2013;45:98-105. https://doi.org/10.1055/s-0032-1325932
  39. Kim SH, Yang DH, Kim JS. Current status of interpretation of small bowel capsule endoscopy. Clin Endosc 2018;51:329-333. https://doi.org/10.5946/ce.2018.095
  40. Nam SJ, Lim YJ, Nam JH, et al. 3D reconstruction of small bowel lesions using stereo camera-based capsule endoscopy. Sci Rep 2020;10:6025.
  41. Oh DJ, Kim KS, Lim YJ. A new active locomotion capsule endoscopy under magnetic control and automated reading program. Clin Endosc 2020;53:395-401. https://doi.org/10.5946/ce.2020.127
  42. Milluzzo SM, Cesaro P, Grazioli LM, et al. Artificial Intelligence in lower gastrointestinal endoscopy: the current status and future perspective. Clin Endosc 2021;54:329-339. https://doi.org/10.5946/ce.2020.082
  43. Stidham RW, Liu W, Bishu S, et al. Performance of a deep learning model vs human reviewers in grading endoscopic disease severity of patients with ulcerative colitis. JAMA Netw Open 2019;2:e193963.
  44. Maeda Y, Kudo SE, Mori Y, et al. Fully automated diagnostic system with artificial intelligence using endocytoscopy to identify the presence of histologic inflammation associated with ulcerative colitis (with video). Gastrointest Endosc 2019;89:408-415. https://doi.org/10.1016/j.gie.2018.09.024
  45. Ozawa T, Ishihara S, Fujishiro M, et al. Novel computer-assisted diagnosis system for endoscopic disease activity in patients with ulcerative colitis. Gastrointest Endosc 2019;89:416-421.e1. https://doi.org/10.1016/j.gie.2018.10.020
  46. Liu Y, Chen PC, Krause J, et al. How to read articles that use machine learning: users' guides to the medical literature. JAMA 2019;322:1806-1816. https://doi.org/10.1001/jama.2019.16489
  47. Eelbode T, Sinonquel P, Maes F, et al. Pitfalls in training and validation of deep learning systems. Best Pract Res Clin Gastroenterol 2021;52-53:101712.
  48. der Pol CBV, Tang A. Imaging database preparation for machine learning. Can Assoc Radiol J 2021;72:9-10. https://doi.org/10.1177/0846537120967720
  49. Sutton RA, Sharma P. Overcoming barriers to implementation of artificial intelligence in gastroenterology. Best Pract Res Clin Gastroenterol 2021;52-53:101732.
  50. Pannala R, Krishnan K, Melson J, et al. Artificial intelligence in gastrointestinal endoscopy. VideoGIE 2020;5:598-613. https://doi.org/10.1016/j.vgie.2020.08.013
  51. Berzin TM, Topol EJ. Adding artificial intelligence to gastrointestinal endoscopy. Lancet 2020;395:485.