Browse > Article
http://dx.doi.org/10.14372/IEMEK.2021.16.2.51

Development of AI Service with Surgical Tools Segmentation and Action Recognition  

Choi, Jaehyeop (Kyungpook National University)
Lee, Haejin (Kyungpook National University)
Jeong, Chang Wook (Seoul National University Hospital)
Jung, Heechul (Kyungpook National University)
Publication Information
IEMEK Journal of Embedded Systems and Applications / v.16, no.2, 2021 , pp. 51-57 More about this Journal
Abstract
In this paper, we propose an artificial intelligence (AI) service that plays a supportive role in robot assisted-surgery using deep learning algorithm that have recently been spotlighted in several fields. The proposed AI service is equipped with the ability to segment surgical tools and the ability to recognize the behavior of surgical tools. In addition, such AI service is opened using public web page to make them easier for surgeons to use. Models mounted on AI service are segmentation deep learning model and action recognition deep learning model. The segmentation deep learning model showed a final mIoU performance of 0.867 for seven surgical tools, and the action recognition deep learning model shows an accuracy of 86.96% for the opening and closing actions of all surgical tools.
Keywords
Deep learning; Convolutional neural network (CNN); Surgical tool; Segmentation; Recognition; AI service;
Citations & Related Records
연도 인용수 순위
  • Reference
1 Saining Xie, Ross Girshick, Piotr Dollar, Zhuowen Tu, Kaiming He. "Aggregated residual transformations for deep neural networks." Proceedings of the IEEE conference on computer vision and pattern recognition. 2017.
2 Zagoruyko, Sergey, Nikos Komodakis. "Wide residual networks." arXiv preprint arXiv:1605.07146 (2016).
3 Bryan C. Russell, Antonio Torralba, Kevin P. Murphy, William T. Freeman. "LabelMe: a database and web-based tool for image annotation." International journal of computer vision 77.1-3 (2008): 157-173.   DOI
4 Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg and Li Fei-Fei "Imagenet large scale visual recognition challenge." International journal of computer vision 115.3 (2015): 211-252.   DOI
5 Krizhevsky, Alex, Ilya Sutskever, Geoffrey E. Hinton. "ImageNet classification with deep convolutional neural networks." Communications of the ACM 60.6 (2017): 84-90.   DOI
6 K. He, X. Zhang, S. Ren, J. Sun, "Deep residual learning for image recognition," IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770-778, 2016.
7 Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, Andrew Rabinovich. "Going deeper with convolutions." Proceedings of the IEEE conference on computer vision and pattern recognition. 2015.
8 Gao Huang, Zhuang Liu, Laurens van der Maaten, Kilian Q. Weinberger. "Densely connected convolutional networks." Proceedings of the IEEE conference on computer vision and pattern recognition. 2017.
9 Long, Jonathan, Evan Shelhamer, Trevor Darrell. "Fully convolutional networks for semantic segmentation." Proceedings of the IEEE conference on computer vision and pattern recognition. 2015.
10 Ronneberger, Olaf, Philipp Fischer, Thomas Brox. "U-net: Convolutional networks for biomedical image segmentation." International Conference on Medical image computing and computer-assisted intervention. Springer, Cham, 2015.
11 Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, Alan L. Yuille. "Semantic image segmentation with deep convolutional nets and fully connected crfs." arXiv preprint arXiv:1412.7062 (2014).
12 Simonyan, Karen, Andrew Zisserman. "Very deep convolutional networks for large-scale image recognition." arXiv preprint arXiv:1409.1556 (2014).
13 Liang-Chieh Chen, George Papandreou, Florian Schroff, Hartwig Adam. "Rethinking atrous convolution for semantic image segmentation." arXiv preprint arXiv:1706.05587 (2017).
14 Tan, Mingxing, Quoc Le. "Efficientnet: Rethinking model scaling for convolutional neural networks." International Conference on Machine Learning. PMLR, 2019.
15 Liang-Chieh Chen, Yukun Zhu, George Papandreou, Florian Schroff, Hartwig Adam. "Encoder-decoder with atrous separable convolution for semantic image segmentation." Proceedings of the European conference on computer vision (ECCV). 2018.
16 Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, Alan L. Yuille. "Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs." IEEE transactions on pattern analysis and machine intelligence 40.4 (2017): 834-848.   DOI