The way to make training data for deep learning model to recognize keywords in product catalog image at E-commerce

From the 21st century, various high-quality services have come up with the growth of the internet or 'Information and Communication Technologies'. Especially, the scale of E-commerce industry in which Amazon and E-bay are standing out is exploding in a large way. As E-commerce grows, Customers could get what they want to buy easily while comparing various products because more products have been registered at online shopping malls. However, a problem has arisen with the growth of E-commerce. As too many products have been registered, it has become difficult for customers to search what they really need in the flood of products. When customers search for desired products with a generalized keyword, too many products have come out as a result. On the contrary, few products have been searched if customers type in details of products because concrete product-attributes have been registered rarely. In this situation, recognizing texts in images automatically with a machine can be a solution. Because bulk of product details are written in catalogs as image format, most of product information are not searched with text inputs in the current text-based searching system. It means if information in images can be converted to text format, customers can search products with product-details, which make them shop more conveniently. There are various existing OCR(Optical Character Recognition) programs which can recognize texts in images. But existing OCR programs are hard to be applied to catalog because they have problems in recognizing texts in certain circumstances, like texts are not big enough or fonts are not consistent. Therefore, this research suggests the way to recognize keywords in catalog with the Deep Learning algorithm which is state of the art in image-recognition area from 2010s. Single Shot Multibox Detector(SSD), which is a credited model for object-detection performance, can be used with structures re-designed to take into account the difference of text from object. But there is an issue that SSD model needs a lot of labeled-train data to be trained, because of the characteristic of deep learning algorithms, that it should be trained by supervised-learning. To collect data, we can try labelling location and classification information to texts in catalog manually. But if data are collected manually, many problems would come up. Some keywords would be missed because human can make mistakes while labelling train data. And it becomes too time-consuming to collect train data considering the scale of data needed or costly if a lot of workers are hired to shorten the time. Furthermore, if some specific keywords are needed to be trained, searching images that have the words would be difficult, as well. To solve the data issue, this research developed a program which create train data automatically. This program can make images which have various keywords and pictures like catalog and save location-information of keywords at the same time. With this program, not only data can be collected efficiently, but also the performance of SSD model becomes better. The SSD model recorded 81.99% of recognition rate with 20,000 data created by the program. Moreover, this research had an efficiency test of SSD model according to data differences to analyze what feature of data exert influence upon the performance of recognizing texts in images. As a result, it is figured out that the number of labeled keywords, the addition of overlapped keyword label, the existence of keywords that is not labeled, the spaces among keywords and the differences of background images are related to the performance of SSD model. This test can lead performance improvement of SSD model or other text-recognizing machine based on deep learning algorithm with high-quality data. SSD model which is re-designed to recognize texts in images and the program developed for creating train data are expected to contribute to improvement of searching system in E-commerce. Suppliers can put less time to register keywords for products and customers can search products with product-details which is written on the catalog.

온라인 쇼핑몰에서 상품 설명 이미지 내의 키워드 인식을 위한 딥러닝 훈련 데이터 자동 생성 방안

E-commerce 환경의 발전으로 소비자들은 다양한 상품들을 한 자리에서 폭 넓게 비교할 수 있게 되었다. 하지만 온라인 쇼핑몰에 올라와있는 상당량의 주요 상품 정보들이 이미지 형태이기 때문에 컴퓨터가 인지할 수 있는 텍스트 기반 검색 시스템에 반영될 수 없다는 한계가 존재한다. 이러한 한계점은 일반적으로 기존 기계학습 기술 및 OCR(Optical Character Recognition) 기술을 활용해, 이미지 형태로 된 키워드를 인식함으로써 개선할 수 있다. 그러나 기존 OCR 기술은 이미지 안에 글자가 아닌 그림이 많고 글자 크기가 작으면 낮은 인식률을 보인다는 문제가 있다. 이에 본 연구에서는 기존 기술들의 한계점을 해결하기 위하여, 딥러닝 기반 사물인식 모형 중 하나인 SSD(Single Shot MultiBox Detector)를 개조하여 이미지 형태의 상품 카탈로그 내의 텍스트 인식모형을 설계하였다. 하지만 이를 학습시키기 위한 데이터를 구축하는 데 상당한 시간과 비용이 필요했는데, 이는 지도학습의 방법론을 따르는 SSD 모형은 훈련 데이터마다 직접 정답 라벨링을 해줘야 하기 때문이다. 본 연구는 이러한 문제점을 해결하기 위해 '훈련 데이터 자동 생성 프로그램'을 함께 개발하였다. 훈련 데이터 자동 생성 프로그램을 통해 수작업으로 데이터를 만드는 것에 비하여 시간과 비용을 대폭 절감할 수 있었으며, 생성된 훈련용 데이터를 통해 모형의 인식 성능을 높일 수 있었다. 더 나아가 실험연구를 통해 자동으로 생성된 훈련 데이터의 특징별로 인식기 모형의 성능에 얼마나 큰 영향을 끼치는지 알아보고, 성능 향상에 효과적인 데이터의 특징을 분석하였다. 본 연구를 통해서 개발된 상품 카탈로그 내 텍스트 인식모형과 훈련 데이터 자동 생성 프로그램은 온라인 쇼핑몰 판매자들의 상품 정보 등록 수고를 줄여줄 수 있으며, 구매자들의 상품 검색 시 결과의 정확성을 향상시키는 데 기여할 수 있을 것으로 기대한다.