Clinical Endoscopy

The Korean Society of Gastrointestinal Endoscopy (대한소화기내시경학회)
권호 표지
DOI QR코드

DOI QR Code

Performance comparison between two computer-aided detection colonoscopy models by trainees using different false positive thresholds: a cross-sectional study in Thailand

  • Kasenee Tiankanon (Division of Gastroenterology, Department of Medicine, Faculty of Medicine, Chulalongkorn University and King Chulalongkorn Memorial Hospital) ;
  • Julalak Karuehardsuwan (Division of Gastroenterology, Department of Medicine, Faculty of Medicine, Chulalongkorn University and King Chulalongkorn Memorial Hospital) ;
  • Satimai Aniwan (Division of Gastroenterology, Department of Medicine, Faculty of Medicine, Chulalongkorn University and King Chulalongkorn Memorial Hospital) ;
  • Parit Mekaroonkamol (Division of Gastroenterology, Department of Medicine, Faculty of Medicine, Chulalongkorn University and King Chulalongkorn Memorial Hospital) ;
  • Panukorn Sunthornwechapong (Department of Computer Engineering, Faculty of Engineering, Chulalongkorn University) ;
  • Huttakan Navadurong (Division of Gastroenterology, Department of Medicine, Faculty of Medicine, Chulalongkorn University and King Chulalongkorn Memorial Hospital) ;
  • Kittithat Tantitanawat (Division of Gastroenterology, Department of Medicine, Faculty of Medicine, Chulalongkorn University and King Chulalongkorn Memorial Hospital) ;
  • Krittaya Mekritthikrai (Division of Gastroenterology, Department of Medicine, Faculty of Medicine, Chulalongkorn University and King Chulalongkorn Memorial Hospital) ;
  • Salin Samutrangsi (Division of Gastroenterology, Department of Medicine, Faculty of Medicine, Chulalongkorn University and King Chulalongkorn Memorial Hospital) ;
  • Peerapon Vateekul (Department of Computer Engineering, Faculty of Engineering, Chulalongkorn University) ;
  • Rungsun Rerknimitr (Division of Gastroenterology, Department of Medicine, Faculty of Medicine, Chulalongkorn University and King Chulalongkorn Memorial Hospital)
  • Received : 2023.05.31
  • Accepted : 2023.09.25
  • Published : 2024.03.30
Download PDF
⟨ Previous Next ⟩

Abstract

Background/Aims: This study aims to compare polyp detection performance of "Deep-GI," a newly developed artificial intelligence (AI) model, to a previously validated AI model computer-aided polyp detection (CADe) using various false positive (FP) thresholds and determining the best threshold for each model. Methods: Colonoscopy videos were collected prospectively and reviewed by three expert endoscopists (gold standard), trainees, CADe (CAD EYE; Fujifilm Corp.), and Deep-GI. Polyp detection sensitivity (PDS), polyp miss rates (PMR), and false-positive alarm rates (FPR) were compared among the three groups using different FP thresholds for the duration of bounding boxes appearing on the screen. Results: In total, 170 colonoscopy videos were used in this study. Deep-GI showed the highest PDS (99.4% vs. 85.4% vs. 66.7%, p<0.01) and the lowest PMR (0.6% vs. 14.6% vs. 33.3%, p<0.01) when compared to CADe and trainees, respectively. Compared to CADe, Deep-GI demonstrated lower FPR at FP thresholds of ≥0.5 (12.1 vs. 22.4) and ≥1 second (4.4 vs. 6.8) (both p<0.05). However, when the threshold was raised to ≥1.5 seconds, the FPR became comparable (2 vs. 2.4, p=0.3), while the PMR increased from 2% to 10%. Conclusions: Compared to CADe, Deep-GI demonstrated a higher PDS with significantly lower FPR at ≥0.5- and ≥1-second thresholds. At the ≥1.5-second threshold, both systems showed comparable FPR with increased PMR.

Keywords

Acknowledgement

This project was funded by the National Research Council of Thailand (NRCT; N42A640330) and the Center of Excellence for Gastrointestinal and Oncology Endoscopy Unit, King Chulalongkorn Memorial Hospital.

References

  1. Virani S, Bilheem S, Chansaard W, et al. National and subnational population-based incidence of cancer in Thailand: assessing cancers with the highest burdens. Cancers (Basel) 2017;9:108. 
  2. Siegel RL, Miller KD, Goding Sauer A, et al. Colorectal cancer statistics, 2020. CA Cancer J Clin 2020;70:145-164.  https://doi.org/10.3322/caac.21601
  3. Zauber AG, Winawer SJ, O'Brien MJ, et al. Colonoscopic polypectomy and long-term prevention of colorectal-cancer deaths. N Engl J Med 2012;366:687-696.  https://doi.org/10.1056/NEJMoa1100370
  4. Doubeni CA, Corley DA, Quinn VP, et al. Effectiveness of screening colonoscopy in reducing the risk of death from right and left colon cancer: a large community-based study. Gut 2018;67:291-298.  https://doi.org/10.1136/gutjnl-2016-312712
  5. Rex DK, Schoenfeld PS, Cohen J, et al. Quality indicators for colonoscopy. Am J Gastroenterol 2015;110:72-90.  https://doi.org/10.1038/ajg.2014.385
  6. Corley DA, Jensen CD, Marks AR, et al. Adenoma detection rate and risk of colorectal cancer and death. N Engl J Med 2014;370:1298-1306.  https://doi.org/10.1056/NEJMoa1309086
  7. Rex DK. Colonoscopic withdrawal technique is associated with adenoma miss rates. Gastrointest Endosc 2000;51:33-36.  https://doi.org/10.1016/S0016-5107(00)70383-X
  8. Barclay RL, Vicari JJ, Doughty AS, et al. Colonoscopic withdrawal times and adenoma detection during screening colonoscopy. N Engl J Med 2006;355:2533-2541.  https://doi.org/10.1056/NEJMoa055498
  9. Francis DL, Rodriguez-Correa DT, Buchner A, et al. Application of a conversion factor to estimate the adenoma detection rate from the polyp detection rate. Gastrointest Endosc 2011;73:493-497.  https://doi.org/10.1016/j.gie.2011.01.005
  10. Boroff ES, Gurudu SR, Hentz JG, et al. Polyp and adenoma detection rates in the proximal and distal colon. Am J Gastroenterol 2013;108:993-999.  https://doi.org/10.1038/ajg.2013.68
  11. Niv Y. Polyp detection rate may predict adenoma detection rate: a meta-analysis. Eur J Gastroenterol Hepatol 2018;30:247-251.  https://doi.org/10.1097/MEG.0000000000001062
  12. Ng S, Sreenivasan AK, Pecoriello J, et al. Polyp detection rate correlates strongly with adenoma detection rate in trainee endoscopists. Dig Dis Sci 2020;65:2229-2233.  https://doi.org/10.1007/s10620-020-06049-0
  13. Aziz M, Fatima R, Dong C, et al. The impact of deep convolutional neural network-based artificial intelligence on colonoscopy outcomes: a systematic review with meta-analysis. J Gastroenterol Hepatol 2020;35:1676-1683.  https://doi.org/10.1111/jgh.15070
  14. Wang P, Berzin TM, Glissen Brown JR, et al. Real-time automatic detection system increases colonoscopic polyp and adenoma detection rates: a prospective randomised controlled study. Gut 2019;68:1813-1819.  https://doi.org/10.1136/gutjnl-2018-317500
  15. Urban G, Tripathi P, Alkayali T, et al. Deep learning localizes and identifies polyps in real time with 96% accuracy in screening colonoscopy. Gastroenterology 2018;155:1069-1078.  https://doi.org/10.1053/j.gastro.2018.06.037
  16. Repici A, Badalamenti M, Maselli R, et al. Efficacy of real-time computer-aided detection of colorectal neoplasia in a randomized trial. Gastroenterology 2020;159:512-520.  https://doi.org/10.1053/j.gastro.2020.04.062
  17. Barua I, Vinsard DG, Jodal HC, et al. Artificial intelligence for polyp detection during colonoscopy: a systematic review and meta-analysis. Endoscopy 2021;53:277-284.  https://doi.org/10.1055/a-1201-7165
  18. Kudo SE, Misawa M, Mori Y, et al. Artificial intelligence-assisted system improves endoscopic identification of colorectal neoplasms. Clin Gastroenterol Hepatol 2020;18:1874-1881.  https://doi.org/10.1016/j.cgh.2019.09.009
  19. Mori Y, Bretthauer M. Addressing false-positive findings with artificial intelligence for polyp detection. Endoscopy 2021;53:941-942.  https://doi.org/10.1055/a-1381-7849
  20. Misawa M, Kudo SE, Mori Y, et al. Artificial intelligence-assisted polyp detection for colonoscopy: initial experience. Gastroenterology 2018;154:2027-2029.  https://doi.org/10.1053/j.gastro.2018.04.003
  21. Holzwanger EA, Bilal M, Glissen Brown JR, et al. Benchmarking definitions of false-positive alerts during computer-aided polyp detection in colonoscopy. Endoscopy 2021;53:937-940.  https://doi.org/10.1055/a-1302-2942
  22. Zhan W, Sun C, Wang M, et al. An improved Yolov5 real-time detection method for small objects captured by UAV. Soft Comput 2022;26:361-373.  https://doi.org/10.1007/s00500-021-06407-8
  23. Alagappan M, Brown JR, Mori Y, et al. Artificial intelligence in gastrointestinal endoscopy: the future is almost here. World J Gastrointest Endosc 2018;10:239-249.  https://doi.org/10.4253/wjge.v10.i10.239
  24. Vinsard DG, Mori Y, Misawa M, et al. Quality assurance of computer-aided detection and diagnosis in colonoscopy. Gastrointest Endosc 2019;90:55-63.  https://doi.org/10.1016/j.gie.2019.03.019
  25. Wang Z, Liang Z, Li L, et al. Reduction of false positives by internal features for polyp detection in CT-based virtual colonoscopy. Med Phys 2005;32:3602-3616.  https://doi.org/10.1118/1.2122447
  26. Wang P, Xiao X, Glissen Brown JR, et al. Development and validation of a deep-learning algorithm for the detection of polyps during colonoscopy. Nat Biomed Eng 2018;2:741-748.  https://doi.org/10.1038/s41551-018-0301-3
  27. Weigt J, Repici A, Antonelli G, et al. Performance of a new integrated computer-assisted system (CADe/CADx) for detection and characterization of colorectal neoplasia. Endoscopy 2022;54:180-184.  https://doi.org/10.1055/a-1372-0419
  28. Wang P, Liu X, Berzin TM, et al. Effect of a deep-learning computer-aided detection system on adenoma detection during colonoscopy (CADe-DB trial): a double-blind randomised study. Lancet Gastroenterol Hepatol 2020;5:343-351.  https://doi.org/10.1016/S2468-1253(19)30411-X