Clinical Endoscopy

The Korean Society of Gastrointestinal Endoscopy (대한소화기내시경학회)
권호 표지
DOI QR코드

DOI QR Code

Application of Artificial Intelligence in Capsule Endoscopy: Where Are We Now?

  • Hwang, Youngbae (Intelligent Image Processing Research Center, Korea Electronics Technology Institute (KETI)) ;
  • Park, Junseok (Digestive Disease Center, Institute for Digestive Research, Department of Internal Medicine, Soonchunhyang University College of Medicine) ;
  • Lim, Yun Jeong (Department of Internal Medicine, Dongguk University Ilsan Hospital, Dongguk University College of Medicine) ;
  • Chun, Hoon Jai (Division of Gastroenterology and Hepatology, Department of Internal Medicine, Korea University College of Medicine)
  • Received : 2018.10.07
  • Accepted : 2018.11.02
  • Published : 2018.11.30
⟨ Previous Next ⟩

Abstract

Unlike wired endoscopy, capsule endoscopy requires additional time for a clinical specialist to review the operation and examine the lesions. To reduce the tedious review time and increase the accuracy of medical examinations, various approaches have been reported based on artificial intelligence for computer-aided diagnosis. Recently, deep learning-based approaches have been applied to many possible areas, showing greatly improved performance, especially for image-based recognition and classification. By reviewing recent deep learning-based approaches for clinical applications, we present the current status and future direction of artificial intelligence for capsule endoscopy.

Keywords

Acknowledgement

Grant : Development of 4D reconstruction and dynamic deformable action model-based hyper-realistic service technology

References

  1. Iddan G, Meron G, Glukhovsky A, Swain P. Wireless capsule endoscopy. Nature 2000;405:417.
  2. Fisher LR, Hasler WL. New vision in video capsule endoscopy: current status and future directions. Nat Rev Gastroenterol Hepatol 2012;9:392-405. https://doi.org/10.1038/nrgastro.2012.88
  3. Kwack WG, Lim YJ. Current status and research into overcoming limitations of capsule endoscopy. Clin Endosc 2016;49:8-15.
  4. Szeliski R. Computer vision: algorithms and applications. London: Springer-Verlag; 2011.
  5. Liedlgruber M, Uhl A. Computer-aided decision support systems for endoscopy in the gastrointestinal tract: a review. IEEE Rev Biomed Eng 2011;4:73-88. https://doi.org/10.1109/RBME.2011.2175445
  6. Iakovidis DK, Koulaouzidis A. Software for enhanced video capsule endoscopy: challenges for essential progress. Nat Rev Gastroenterol Hepatol 2015;12:172-186. https://doi.org/10.1038/nrgastro.2015.13
  7. Iakovidis DK, Koulaouzidis A. Automatic lesion detection in capsule endoscopy based on color saliency: closer to an essential adjunct for reviewing software. Gastrointest Endosc 2014;80:877-883. https://doi.org/10.1016/j.gie.2014.06.026
  8. Lv G, Yan G, Wang Z. Bleeding detection in wireless capsule endoscopy images based on color invariants and spatial pyramids using support vector machines. Conf Proc IEEE Eng Med Biol Soc 2011;2011:6643-6646.
  9. Karargyris A, Bourbakis N. Detection of small bowel polyps and ulcers in wireless capsule endoscopy videos. IEEE Trans Biomed Eng 2011;58:2777-2786. https://doi.org/10.1109/TBME.2011.2155064
  10. Pan G, Yan G, Qiu X, Cui J. Bleeding detection in wireless capsule endoscopy based on probabilistic neural network. J Med Syst 2011;35:1477-1484. https://doi.org/10.1007/s10916-009-9424-0
  11. Mamonov AV, Figueiredo IN, Figueiredo PN, Tsai YH. Automated polyp detection in colon capsule endoscopy. IEEE Trans Med Imaging 2014;33:1488-1502. https://doi.org/10.1109/TMI.2014.2314959
  12. Harris C, Stephens M. A combined corner and edge detector. In: Proceedings of the Alvey Vision Conference 1988; 1988 Aug 31-Sep 2; Romsey, UK. Romsey: Roke Manor Research; 1988. p. 147-151.
  13. Canny J. A computational approach to edge detection. IEEE Trans Pattern Anal Mach Intell 1986;8:679-698.
  14. Lowe DG. Object recognition from local scale-invariant features. In: Proceedings of the Seventh IEEE International Conference on Computer Vision; 1999 Sep 20-27; Kerkyra, Greece. Piscataway (NJ): IEEE; 1999. p. 1150-1157.
  15. Bay H, Ess A, Tuytelaars T, Van Gool L. Speeded-up robust features (SURF). Comput Vis Image Underst 2008;110:346-359. https://doi.org/10.1016/j.cviu.2007.09.014
  16. Mikolajczyk K, Schmid C. Scale & affine invariant interest point detectors. Int J Comput Vis 2004;60:63-86. https://doi.org/10.1023/B:VISI.0000027790.02288.f2
  17. Belongie S, Malik J, Puzicha J. Shape matching and object recognition using shape contexts. IEEE Trans Pattern Anal Mach Intell 2002;24:509-522. https://doi.org/10.1109/34.993558
  18. Dalal N, Triggs B. Histograms of oriented gradients for human detection. In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05); 2005 Jun 20-25; San Diego (CA), USA. Piscataway (NJ): IEEE; 2005. p. 886-893.
  19. Felzenszwalb PF, Girshick RB, McAllester D, Ramanan D. Object detection with discriminatively trained part-based models. IEEE Trans Pattern Anal Mach Intell 2010;32:1627-1645. https://doi.org/10.1109/TPAMI.2009.167
  20. Russakovsky O, Deng J, Su H, et al. ImageNet large scale visual recognition challenge. Int J Comput Vis 2015;115:211-252. https://doi.org/10.1007/s11263-015-0816-y
  21. Krizhevsky A, Sutskever I, Hinton GE. ImageNet classification with deep convolutional neural networks. In: NIPS'12 Proceedings of the 25th International Conference on Neural Information Processing Systems; 2012 Dec 3-6; Lake Tahoe (NV), USA. Red Hook (NY): Curran Associates, Inc.; 2012. p. 1097-1105.
  22. Simonyan K, Zisserman A. Very deep convolutional networks for largescale image recognition. ArXiv e-prints 2014. https://ui.adsabs.harvard.edu/#abs/2014arXiv1409.1556S.
  23. Szegedy C, Liu W, Jia Y, et al. Going deeper with convolutions. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR); 2015 Jun 7-12; Boston (MA), USA. Piscataway (NJ): IEEE; 2015. p. 1-9.
  24. He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR); 2016 Jun 27-30; Las Vegas (NV), USA. Piscataway (NJ): IEEE; 2016. p. 770-778.
  25. Urban G, Tripathi P, Alkayali T, et al. Deep learning localizes and identifies polyps in real time with 96% accuracy in screening colonoscopy. Gastroenterology 2018;155:1069-1078.e8. https://doi.org/10.1053/j.gastro.2018.06.037
  26. Redmon J, Divvala S, Girshick R, Farhadi A. You only look once: unified, real-time object detection. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR); 2016 Jun 27-30; Las Vegas (NV), USA. Piscataway (NJ): IEEE; 2016. p. 779-788.
  27. Zou Y, Li L, Wang Y, Yu J, Li Y, Deng WL. Classifying digestive organs in wireless capsule endoscopy images based on deep convolutional neural network. In: 2015 IEEE International Conference on Digital Signal Processing (DSP); 2015 Jul 21-24; Singapore. Piscataway (NJ): IEEE; 2015. p. 1274-1278.
  28. Segui S, Drozdzal M, Pascual G, et al. Generic feature learning for wireless capsule endoscopy analysis. Comput Biol Med 2016;79:163-172. https://doi.org/10.1016/j.compbiomed.2016.10.011
  29. Jia X, Meng MQH. A deep convolutional neural network for bleeding detection in wireless capsule endoscopy images. In: 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC); 2016 Aug 16-20; Orlando (FL), USA. Piscataway (NJ): IEEE; 2016. p. 639-642.
  30. Li P, Li Z, Gao F, Wan L, Yu J. Convolutional neural networks for intestinal hemorrhage detection in wireless capsule endoscopy images. In: 2017 IEEE International Conference on Multimedia and Expo (ICME); 2017 Jul 10-14; Hong Kong, China. Piscataway (NJ): IEEE; 2017. p. 1518-1523.
  31. Leenhardt R, Vasseur P, Li C, et al. A neural network algorithm for detection of GI angiectasia during small-bowel capsule endoscopy. Gastrointest Endosc 2018 Jul 11 [Epub]. https://doi.org/10.1016/j.gie.2018.06.036.
  32. Yuan Y, Meng MQ. Deep learning for polyp recognition in wireless capsule endoscopy images. Med Phys 2017;44:1379-1389. https://doi.org/10.1002/mp.12147
  33. He JY, Wu X, Jiang YG, Peng Q, Jain R. Hookworm detection in wireless capsule endoscopy images with deep learning. IEEE Trans Image Process 2018;27:2379-2392. https://doi.org/10.1109/TIP.2018.2801119
  34. Iakovidis DK, Georgakopoulos SV, Vasilakakis M, Koulaouzidis A, Plagianakos VP. Detecting and locating gastrointestinal anomalies using deep learning and iterative cluster unification. IEEE Trans Med Imaging 2018;37:2196-2210. https://doi.org/10.1109/TMI.2018.2837002
  35. Oliva A, Torralba A. Modeling the shape of the scene: a holistic representation of the spatial envelope. Int J Comput Vis 2001;42:145-175. https://doi.org/10.1023/A:1011139631724
  36. Khan S, Yong SP. A comparison of deep learning and hand crafted features in medical image modality classification. In: 2016 3rd International Conference on Computer and Information Sciences (ICCOINS); 2016 Aug 15-17; Kuala Lumpur, Malaysia. Piscataway (NJ): IEEE; 2016. p. 633-638.
  37. Ojala T, Pietikainen M, Maenpaa T. Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans Pattern Anal Mach Intell 2002;24:971-987. https://doi.org/10.1109/TPAMI.2002.1017623
  38. Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R. Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 2014;15:1929-1958.
  39. Cogswell M, Ahmed F, Girshick R, Zitnick L, Batra D. Reducing overfitting in deep networks by decorrelating representations. ArXiv e-prints 2015. http://adsabs.harvard.edu/cgi-bin/bib_query?arXiv:1511.06068.
  40. Kominami Y, Yoshida S, Tanaka S, et al. Computer-aided diagnosis of colorectal polyp histology by using a real-time image recognition system and narrow-band imaging magnifying colonoscopy. Gastrointest Endosc 2016;83:643-649. https://doi.org/10.1016/j.gie.2015.08.004
  41. Mori Y, Kudo SE, Misawa M, et al. Real-time use of artificial intelligence in identification of diminutive polyps during colonoscopy: a prospective study. Ann Intern Med 2018;169:357-366. https://doi.org/10.7326/M18-0249

Cited by

  1. Recent Development of Computer Vision Technology to Improve Capsule Endoscopy vol.52, pp.4, 2018, https://doi.org/10.5946/ce.2018.172
  2. Deep learning algorithms for automated detection of Crohn’s disease ulcers by video capsule endoscopy vol.91, pp.3, 2018, https://doi.org/10.1016/j.gie.2019.11.012
  3. Artificial Intelligence in Gastrointestinal Endoscopy vol.53, pp.2, 2018, https://doi.org/10.5946/ce.2020.038
  4. Robotics in the Gut vol.3, pp.4, 2018, https://doi.org/10.1002/adtp.201900125
  5. Editors' Choice of Noteworthy Clinical Endoscopy Publications in the First Decade vol.54, pp.5, 2018, https://doi.org/10.5946/ce.2021.216
  6. Applicability of colon capsule endoscopy as pan-endoscopy: From bowel preparation, transit, and rating times to completion rate and patient acceptance vol.9, pp.12, 2021, https://doi.org/10.1055/a-1578-1800