Journal of Intelligence and Information Systems (지능정보연구)

Korea Intelligent Information System Society (한국지능정보시스템학회)
권호 표지
DOI QR코드

DOI QR Code

Deep Learning OCR based document processing platform and its application in financial domain

금융 특화 딥러닝 광학문자인식 기반 문서 처리 플랫폼 구축 및 금융권 내 활용

  • Received : 2022.11.14
  • Accepted : 2022.12.23
  • Published : 2023.03.31
Download PDF
⟨ Previous Next ⟩

Abstract

With the development of deep learning technologies, Artificial Intelligence powered Optical Character Recognition (AI-OCR) has evolved to read multiple languages from various forms of images accurately. For the financial industry, where a large number of diverse documents are processed through manpower, the potential for using AI-OCR is great. In this study, we present a configuration and a design of an AI-OCR modality for use in the financial industry and discuss the platform construction with application cases. Since the use of financial domain data is prohibited under the Personal Information Protection Act, we developed a deep learning-based data generation approach and used it to train the AI-OCR models. The AI-OCR models are trained for image preprocessing, text recognition, and language processing and are configured as a microservice architected platform to process a broad variety of documents. We have demonstrated the AI-OCR platform by applying it to financial domain tasks of document sorting, document verification, and typing assistance The demonstrations confirm the increasing work efficiency and conveniences.

인공지능의 발전과 함께 딥러닝을 활용한 인공지능 광학문자인식 기법 (Artificial Intelligence powered Optical Character Recognition, AI-OCR) 의 등장은 기존의 이미지 처리 기반 OCR 기술의 한계를 넘어 다양한 형태의 이미지로부터 여러 언어를 높은 정확도로 읽어낼 수 있는 모델로 발전하였다. 특히, AI-OCR은 인력을 통해 대량의 다양한 서류 처리 업무를 수행하는 금융업에 있어 그 활용 잠재력이 크다. 본 연구에서는 금융권내 활용을 위한 AI-OCR 모델의 구성과 설계를 제시하고, 이를 효율적으로 적용하기 위한 플랫폼 구축 및 활용 사례에 대해 논한다. 금융권 특화 딥러닝 모델을 만듦에 있어 금융 도메인 데이터 사용은 필수적이나, 개인정보보호법 이하 실 데이터의 사용이 불가하다. 이에 본 연구에서는 딥러닝 기반 데이터 생성 모델을 개발하였고, 이를 활용하여 AI-OCR 모델 학습을 진행하였다. 다양한 서류 처리에 있어 유연한 데이터 처리를 위해 단계적 구성의 AI-OCR 모델들을 제안하며, 이는 이미지 전처리 모델, 문자 탐지 모델, 문자 인식 모델, 문자 정렬 모델 및 언어 처리 모델의 선택적, 단계적 사용을 포함한다. AI-OCR 모델의 배포를 위해 온프레미스(On-Premise) 및 프라이빗 클라우드(Private Cloud) 내 GPU 컴퓨팅 클러스터를 구성하고, Hybrid GPU Cluster 내 컨테이너 오케스트레이션을 통한 고효율, 고가용 AI-OCR 플랫폼 구축하여 다양한 업무 및 채널에 적용하였다. 본 연구를 통해 금융 특화 AI-OCR 모델 및 플랫폼을 구축하여 금융권 서류 처리 업무인 문서 분류, 문서 검증 및 입력 보조 시스템으로의 활용을 통해 업무 효율 및 편의성 증대를 확인하였다.

Keywords

References

  1. 김동규, 이동욱, 박장원, 오성우, 권성준, 이인용, 최동원. (2022), KB-BERT: 금융 특화 한국어 사전학습 언어모델과 그 응용, 지능정보연구 28(2), 191-206. https://doi.org/10.13088/JIIS.2022.28.2.191
  2. 이정선, 서보일, 권영욱. (2021), 인공지능이 의사결정에 미치는 영향에 관한 연구: 인간과 인공지능의 협업 및 의사결정자의 성격 특성을 중심으로, 지능정보연구 27(3), 231-252. https://doi.org/10.13088/JIIS.2021.27.3.231
  3. 최은주, 이준영, 한인구. (2020), 딥러닝 오픈소스 프레임워크의 사례연구를 통한 도입 전략도출, 지능정보연구 26(4), 27-65. https://doi.org/10.13088/JIIS.2020.26.4.027
  4. Abdollahi Vayghan, L., Saied, M. A., Toeroe, M., & Khendek, F. (2019). Microservice Based Architecture: Towards High-Availability for Stateful Applications with Kubernetes. 2019 IEEE 19th International Conference on Software Quality, Reliability and Security (QRS), 176-185. https://doi.org/10.1109/QRS.2019.00034
  5. Akyurek, E., Dayanik, E., & Yuret, D. (2019). Morphological Analysis Using a Sequence Decoder. Transactions of the Association for Computational Linguistics, 7, 567-579. https://doi.org/10.1162/tacl_a_00286
  6. Al-Badr, B., & Mahmoud, S. A. (1995). Survey and bibliography of Arabic optical text recognition. Signal Processing, 41(1), 49-77. https://doi.org/10.1016/0165-1684(94)00090-M
  7. Appalaraju, S., Jasani, B., Kota, B. U., Xie, Y., & Manmatha, R. (2021). DocFormer: End-to-End Transformer for Document Understanding. 2021 IEEE/CVF International Conference on Computer Vision (ICCV), 973-983. https://doi.org/10.1109/ICCV48922.2021.00103
  8. Armenise, V. (2015). Continuous Delivery with Jenkins: Jenkins Solutions to Implement Continuous Delivery. 2015 IEEE/ACM 3rd International Workshop on Release Engineering, 24-27. https://doi.org/10.1109/RELENG.2015.19
  9. Atienza, R. (2021). Vision Transformer for Fast and Efficient Scene Text Recognition (pp. 319-334). https://doi.org/10.1007/978-3-030-86549-8_21
  10. Baek, J., Kim, G., Lee, J., Park, S., Han, D., Yun, S., Oh, S. J., & Lee, H. (2019). What Is Wrong With Scene Text Recognition Model Comparisons? Dataset and Model Analysis.
  11. Baek, Y., Lee, B., Han, D., Yun, S., & Lee, H. (2019). Character Region Awareness for Text Detection. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 9357-9366. https://doi.org/10.1109/CVPR.2019.00959
  12. Bartz, C., Yang, H., & Meinel, C. (2017). STN-OCR: A single Neural Network for Text Detection and Text Recognition.
  13. Bisong, E. (2019). Kubeflow and Kubeflow Pipelines. In Building Machine Learning and Deep Learning Models on Google Cloud Platform (pp. 671-685). Apress. https://doi.org/10.1007/978-1-4842-4470-8_46
  14. Bookstein, F. L. (1989). Principal warps: thin-plate splines and the decomposition of deformations. IEEE Transactions on Pattern Analysis and Machine Intelligence, 11(6), 567-585. https://doi.org/10.1109/34.24792
  15. Chauhan, R., Ghanshala, K. K., & Joshi, R. C. (2018). Convolutional Neural Network (CNN) for Image Detection and Recognition. 2018 First International Conference on Secure Cyber Computing and Communication (ICSCCC), 278-282. https://doi.org/10.1109/ICSCCC.2018.8703316
  16. Chen, A., Chow, A., Davidson, A., DCunha, A., Ghodsi, A., Hong, S. A., Konwinski, A., Mewald, C., Murching, S., Nykodym, T., Ogilvie, P., Parkhe, M., Singh, A., Xie, F., Zaharia, M., Zang, R., Zheng, J., & Zumar, C. (2020). Developments in MLflow. Proceedings of the Fourth International Workshop on Data Management for End-to-End Machine Learning, 1-4. https://doi.org/10.1145/3399579.3399867
  17. Cortes Rios, J. C., Kopec-Harding, K., Eraslan, S., Page, C., Haines, R., Jay, C., & Embury, S. M. (2019). A Methodology for Using GitLab for Software Engineering Learning Analytics. 2019 IEEE/ACM 12th International Workshop on Cooperative and Human Aspects of Software Engineering (CHASE), 3-6. https://doi.org/10.1109/CHASE.2019.00009
  18. Creswell, A., White, T., Dumoulin, V., Arulkumaran, K., Sengupta, B., & Bharath, A. A. (2018). Generative Adversarial Networks: An Overview. IEEE Signal Processing Magazine, 35(1), 53-65. https://doi.org/10.1109/MSP.2017.2765202
  19. Dieleman, S., Willett, K. W., & Dambre, J. (2015). Rotation-invariant convolutional neural networks for galaxy morphology prediction. Monthly Notices of the Royal Astronomical Society, 450(2), 1441-1459. https://doi.org/10.1093/mnras/stv632
  20. Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., & Houlsby, N. (2020). An Image is Worth 16×16 Words: Transformers for Image Recognition at Scale.
  21. Ganis, M. D., Wilson, C. L., & Blue, J. L. (1998). Neural network-based systems for handprint OCR applications. IEEE Transactions on Image Processing, 7(8), 1097-1112. https://doi.org/10.1109/83.704304
  22. Gatys, L. A., Ecker, A. S., & Bethge, M. (2016). Image Style Transfer Using Convolutional Neural Networks. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2414-2423. https://doi.org/10.1109/CVPR.2016.265
  23. Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014). Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation. 2014 IEEE Conference on Computer Vision and Pattern Recognition, 580-587. https://doi.org/10.1109/CVPR.2014.81
  24. Gos, K., & Zabierowski, W. (2020). The Comparison of Microservice and Monolithic Architecture. 2020 IEEE XVIth International Conference on the Perspective Technologies and Methods in MEMS Design (MEMSTECH), 150-153. https://doi.org/10.1109/MEMSTECH49584.2020.9109514
  25. Graves, A., Fernandez, S., Gomez, F., & Schmidhuber, J. (2006). Connectionist temporal classification. Proceedings of the 23rd International Conference on Machine Learning - ICML '06, 369-376. https://doi.org/10.1145/1143844.1143891
  26. Hewage, N., & Meedeniya, D. (2022). Machine Learning Operations: A Survey on MLOps Tool Support. https://doi.org/10.48550/arXiv.2202.10169
  27. Hochreiter, S., & Schmidhuber, J. (1997). Long Short-Term Memory. Neural Computation, 9(8), 1735-1780. https://doi.org/10.1162/neco.1997.9.8.1735
  28. Howard, J., & Ruder, S. (2018). Universal Language Model Fine-tuning for Text Classification. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 328-339. https://doi.org/10.18653/v1/P18-1031
  29. Huang, Z., Chen, K., He, J., Bai, X., Karatzas, D., Lu, S., & Jawahar, C. v. (2019). ICDAR2019 Competition on Scanned Receipt OCR and Information Extraction. 2019 International Conference on Document Analysis and Recognition (ICDAR), 1516-1520. https://doi.org/10.1109/ICDAR.2019.00244
  30. Kim, D., Kwak, M., Won, E., Shin, S., & Nam, J. (2020). TLGAN: document Text Localization using Generative Adversarial Nets.
  31. Kim, G., Hong, T., Yim, M., Nam, J., Park, J., Yim, J., Hwang, W., Yun, S., Han, D., & Park, S. (2022). OCR-Free Document Understanding Transformer (pp. 498-517). https://doi.org/10.1007/978-3-031-19815-1_29
  32. Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017). ImageNet classification with deep convolutional neural networks. Communications of the ACM, 60(6), 84-90. https://doi.org/10.1145/3065386
  33. Lample, G., Ballesteros, M., Subramanian, S., Kawakami, K., & Dyer, C. (2016). Neural Architectures for Named Entity Recognition.
  34. Landau, H. J. (1967). Sampling, data transmission, and the Nyquist rate. Proceedings of the IEEE, 55(10), 1701-1706. https://doi.org/10.1109/PROC.1967.5962
  35. Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11), 2278-2324. https://doi.org/10.1109/5.726791
  36. Li Deng. (2012). The MNIST Database of Handwritten Digit Images for Machine Learning Research [Best of the Web]. IEEE Signal Processing Magazine, 29(6), 141-142. https://doi.org/10.1109/MSP.2012.2211477
  37. Li, J., Xu, Y., Lv, T., Cui, L., Zhang, C., & Wei, F. (2022). DiT: Self-supervised Pre-training for Document Image Transformer.
  38. Liu, X., Shen, Y., Duh, K., & Gao, J. (2018). Stochastic Answer Networks for Machine Reading Comprehension. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 1694-1704. https://doi.org/10.18653/v1/P18-1157
  39. Luo, X., Han, Z., Yang, L., & Zhang, L. (2022). Consistent Style Transfer. ArXiv:2201.02233 (2022).
  40. Luong, M.-T., Pham, H., & Manning, C. D. (2015). Effective Approaches to Attention-based Neural Machine Translation.
  41. Mori, S., Suen, C. Y., & Yamamoto, K. (1992). Historical review of OCR research and development. Proceedings of the IEEE, 80(7), 1029-1058. https://doi.org/10.1109/5.156468
  42. Ramadoni, Utami, E., & Fatta, H. al. (2021). Analysis on the Use of Declarative and Pull-based Deployment Models on GitOps Using Argo CD. 2021 4th International Conference on Information and Communications Technology (ICOIACT), 186-191. https://doi.org/10.1109/ICOIACT53268.2021.9563984
  43. Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016). You Only Look Once: Unified, Real-Time Object Detection. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 779-788. https://doi.org/10.1109/CVPR.2016.91
  44. Riazi, M. S., Darvish Rouani, B., & Koushanfar, F. (2019). Deep Learning on Private Data. IEEE Security & Privacy, 17(6), 54-63. https://doi.org/10.1109/MSEC.2019.2935666
  45. Ronneberger, O., Fischer, P., & Brox, T. (2015). U-Net: Convolutional Networks for Biomedical Image Segmentation (pp. 234-241). https://doi.org/10.1007/978-3-319-24574-4_28
  46. Shelhamer, E., Long, J., & Darrell, T. (2017). Fully Convolutional Networks for Semantic Segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(4), 640-651. https://doi.org/10.1109/TPAMI.2016.2572683
  47. Sherstinsky, A. (2020). Fundamentals of Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) network. Physica D: Nonlinear Phenomena, 404, 132306. https://doi.org/10.1016/j.physd.2019.132306
  48. Simonyan, K., & Zisserman, A. (2014). Very Deep Convolutional Networks for Large-Scale Image Recognition.
  49. Singh, A., Bacchuwar, K., & Bhasin, A. (2012). A Survey of OCR Applications. International Journal of Machine Learning and Computing, 314-318. https://doi.org/10.7763/IJMLC.2012.V2.137
  50. Subramani, N., Matton, A., Greaves, M., & Lam, A. (2020). A Survey of Deep Learning Approaches for OCR and Document Understanding.
  51. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017). Attention Is All You Need.
  52. xie, xuemei, Cao, G., Yang, W., Liao, Q., Shi, G., & Wu, J. (2018). Feature-fused SSD: fast detection for small objects. In H. Yu & J. Dong (Eds.), Ninth International Conference on Graphic and Image Processing (ICGIP 2017) (p. 236). SPIE. https://doi.org/10.1117/12.2304811
  53. Xu, Y., Li, M., Cui, L., Huang, S., Wei, F., & Zhou, M. (2020). LayoutLM: Pre-training of Text and Layout for Document Image Understanding. Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 1192-1200. https://doi.org/10.1145/3394486.3403172
  54. Zhu, J.-Y., Park, T., Isola, P., & Efros, A. A. (2017). Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks. 2017 IEEE International Conference on Computer Vision (ICCV), 2242-2251. https://doi.org/10.1109/ICCV.2017.244