Journal of Biomedical Engineering Research (대한의용생체공학회:의공학회지)

The Korean Society of Medical and Biological Engineering (대한의용생체공학회)
권호 표지
DOI QR코드

DOI QR Code

Pill Identification Algorithm Based on Deep Learning Using Imprinted Text Feature

음각 정보를 이용한 딥러닝 기반의 알약 식별 알고리즘 연구

  • Seon Min, Lee (Department of Nursing, College of Nursing, Gachon University) ;
  • Young Jae, Kim (Department of Biomedical Engineering, Gill Medical Center, College of Medicine, Gachon University) ;
  • Kwang Gi, Kim (Department of Health Sciences & Technology, Gachon Advanced Institute for Health Sciences & Tecnology (GAIHST), Gachon University)
  • 이선민 (가천대학교 간호대학 간호학과) ;
  • 김영재 (가천대학교 의과대학 의공학교실) ;
  • 김광기 (가천융합의과학원 융합의과학과)
  • Received : 2022.11.30
  • Accepted : 2022.12.23
  • Published : 2022.12.31
Download PDF
⟨ Previous Next ⟩

Abstract

In this paper, we propose a pill identification model using engraved text feature and image feature such as shape and color, and compare it with an identification model that does not use engraved text feature to verify the possibility of improving identification performance by improving recognition rate of the engraved text. The data consisted of 100 classes and used 10 images per class. The engraved text feature was acquired through Keras OCR based on deep learning and 1D CNN, and the image feature was acquired through 2D CNN. According to the identification results, the accuracy of the text recognition model was 90%. The accuracy of the comparative model and the proposed model was 91.9% and 97.6%. The accuracy, precision, recall, and F1-score of the proposed model were better than those of the comparative model in terms of statistical significance. As a result, we confirmed that the expansion of the range of feature improved the performance of the identification model.

Keywords

Acknowledgement

본 연구는 중소벤처기업부의 맞춤형 기술파트너 지원사업(RS-2022-00165316)과 경기도의 경기도 지역협력연구센터 사업[GRRC-가천 2020(B01), AI기반 의료영상분석]과, 과학기술정보통신부 및 정보통신기획평가원의 대학ICT연구센터육성지원사업의 연구결과로 수행되었음(IITP-2022-2017-0-01630).

References

  1. Oh DE, Lee MS, Lee YD, Ye KN, and Kim JT. The reduction of unreturned medication by improving the various medication return activities. Journal of Korean Society of Healthsystem Pharmacists. 2011;28(4):364-371. https://doi.org/10.32429/jkshp.2011.28.4.003
  2. Eoum GH, Lee JY, Cho YH, Cho YS, Hahn HJ, and Son IJ. The Effect for Pharmacy Intervention on the Decrease of Returned Medications from the Ward. Journal of Korean Society of Health-system Pharmacists. 2006;23(3):1-9.
  3. Yi GY, Kim YJ, Kim ST, Kim HE, and Kim KG. Comparison and Verification of Deep Learning Models for Automatic Recognition of Pills. Journal of Korea Multimedia Society. 2019;22(3):349-356. https://doi.org/10.9717/KMMS.2019.22.3.349
  4. Wang Y, Ribera J, Liu C, Yarlagadda S, Zhu F. Pill Recognition Using Minimal Labeled Data. 2017 IEEE Third International Conference on Multimedia Big Data (BigMM). 2017;346-53.
  5. Larios Delgado N, Usuyama N, Hall AK, Hazen RJ, Ma M, Sahu S, et al. Fast and accurate medication identification. NPJ digital medicine. 2019;2(1):1-9. https://doi.org/10.1038/s41746-018-0076-7
  6. Ou YY, Tsai AC, Zhou XP, Wang JF. Automatic drug pills detection based on enhanced feature pyramid network and convolution neural networks. IET Computer Vision. 2020;14(1):9-17. https://doi.org/10.1049/iet-cvi.2019.0171
  7. Suntronsuk S, Ratanotayanon S, editors. Automatic text imprint analysis from pill images. 2017 9th International Conference on Knowledge and Smart Technology (KST). 2017; 288-293.
  8. Kim DW. Shape and Text Imprint Recognition of Pill Image Taken with a Smartphone. Master's Thesis of Seoul National University. 2017.
  9. Ko DG. Optical Character Recognition Performance Comparison of CNNs and Tesseract. Master's Thesis of Sungkyunkwan University, 2016. [Online]. Available: http://www.riss.kr/link?id=T14177015
  10. Kim YJ and Kim KG. Development of an Optimized Deep Learning Model for Medical Imaging. Journal of the Korean Society of Radiology. 2020;81(6):1274-1289.
  11. Hassanein A.S, Mohammad S, Sameer M, Ragab M.E. A survey on Hough transform, theory, techniques and applications. International Journal of Computer Science Issues (IJCSI). 2015;12(1):139-156.
  12. Xu Z, Baojie X, Guoxin W. Canny edge detection based on Open CV. 2017 13th IEEE international conference on electronic measurement & instruments (ICEMI), 2017;53-56.
  13. Firoz R, Ali M.S, Khan M.N.U, Hossain M.K, Islam M.K, Shahinuzzaman M. Medical image enhancement using morphological transformation. Journal of Data Analysis and Information Processing. 2016;4(1):1-12. https://doi.org/10.4236/jdaip.2016.41001
  14. Erkan U, Gokrem L, Enginoglu S. Different applied median filter in salt and pepper noise. Computers & Electrical Engineering, 2018;70:789-798. https://doi.org/10.1016/j.compeleceng.2018.01.019
  15. Sahu S, Singh A.K, Ghrera S, Elhoseny M. An approach for de-noising and contrast enhancement of retinal fundus image using CLAHE. Optics & Laser Technology, 2019;110:87-98. https://doi.org/10.1016/j.optlastec.2018.06.061
  16. Baek Y, Lee B, Han D, Yun S, Lee H. Character region awareness for text detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019;9365-9374.
  17. Lei Z, Zhao S, Song H, Shen J. Scene text recognition using residual convolutional recurrent neural network. Machine Vision and Applications. 2018;29(5):861-871.
  18. Chotivatunyu P, Hnoohom N. Medicine Identification System on Mobile Devices for the Elderly. 2020 15th International Joint Symposium on Artificial Intelligence and Natural Language Processing (iSAI-NLP). 2020;1-6.
  19. Maitrichit N, Hnoohom N. Intelligent Medicine Identification System Using a Combination of Image Recognition and Optical Character Recognition. 2020 15th International Joint Symposium on Artificial Intelligence and Natural Language Processing (iSAI-NLP). 2020;1-5.
  20. Manaswi NK. Understanding and working with Keras. Deep Learning with Applications Using Python. Berkeley, CA: Apress: 2018;31-43.
  21. Chicho BT, Sallow AB. A Comprehensive Survey of Deep Learning Models Based on Keras Framework. Journal of Soft Computing and Data Mining. 2021;2(2):49-62.
  22. Chollet F, Deep learning with Python. New York: Simon and Schuster: 2021;62-64.
  23. Gulli A, Kapoor A, Pal S. Deep learning with TensorFlow 2 and Keras: regression, ConvNets, GANs, RNNs, NLP, and more with TensorFlow 2 and the Keras API. Birmingham, UK: Packt Publishing Ltd: 2019;63-66.
  24. Albawi S, Mohammed TA, Al-Zawi S. Understanding of a convolutional neural network. 2017 international conference on engineering and technology (ICET). 2017;1-6.
  25. Sun M, Song Z, Jiang X, Pan J, Pang Y. Learning pooling for convolutional neural network. Neurocomputing. 2017;224:96-104. https://doi.org/10.1016/j.neucom.2016.10.049
  26. Vijayarani S, Janani R. Text mining: open source tokenization tools-an analysis. Advanced Computational Intelligence: An International Journal (ACII), 2016;3(1):37-47. https://doi.org/10.5121/acii.2016.3104
  27. Kudo T, Richardson J. Sentencepiece: A simple and language independent subword tokenizer and detokenizer for neural text processing. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, 2018;66-71.
  28. Dwarampudi M, Reddy N. Effects of padding on LSTMs and CNNs. arXiv preprint arXiv:1903.07288, 2019.
  29. Du C, Wang Y, Wang C, Shi C, Xiao B. Selective feature connection mechanism: Concatenating multi-layer CNN features with a feature selector. Pattern Recognition Letters. 2020;129:108-14. https://doi.org/10.1016/j.patrec.2019.11.015
  30. Ertam F, Aydin G. Data classification with deep learning using Tensorflow. 2017 international conference on computer science and engineering (UBMK), 2017;755-758.
  31. Yacouby R, Axman D. Probabilistic extension of precision, recall, and F1 score for more thorough evaluation of classification models. Proceedings of the first workshop on evaluation and comparison of NLP systems. 2020;79-91.