Journal of Korea Society of Industrial Information Systems (한국산업정보학회논문지)

Korea Society of Industrial Information Systems (한국산업정보학회)
권호 표지
DOI QR코드

DOI QR Code

Generating Pairwise Comparison Set for Crowed Sourcing based Deep Learning

크라우드 소싱 기반 딥러닝 선호 학습을 위한 쌍체 비교 셋 생성

  • 유기현 (군산대학교 컴퓨터정보통신공학부) ;
  • 이동기 (군산대학교 컴퓨터정보통신공학부) ;
  • 이창우 (군산대학교 컴퓨터정보통신공학부) ;
  • 남광우 (군산대학교 컴퓨터정보통신공학부)
  • Received : 2022.08.23
  • Accepted : 2022.09.26
  • Published : 2022.10.30
Download PDF
⟨ Previous Next ⟩

Abstract

With the development of deep learning technology, various research and development are underway to estimate preference rankings through learning, and it is used in various fields such as web search, gene classification, recommendation system, and image search. Approximation algorithms are used to estimate deep learning-based preference ranking, which builds more than k comparison sets on all comparison targets to ensure proper accuracy, and how to build comparison sets affects learning. In this paper, we propose a k-disjoint comparison set generation algorithm and a k-chain comparison set generation algorithm, a novel algorithm for generating paired comparison sets for crowd-sourcing-based deep learning affinity measurements. In particular, the experiment confirmed that the k-chaining algorithm, like the conventional circular generation algorithm, also has a random nature that can support stable preference evaluation while ensuring connectivity between data.

딥러닝 기술의 발전에 따라 학습을 통해 선호도 랭킹 추정을 하기 위한 다양한 연구 개발이 진행되고 있으며, 웹 검색, 유전자 분류, 추천 시스템, 이미지 검색 등 여러 분야에 걸쳐 이용되고 있다. 딥러닝 기반의 선호도 랭킹을 추정하기 위해 근사(approximation) 알고리즘을 이용하는데, 이 근사 알고리즘에서 적정한 정도의 정확도를 보장할 수 있도록 모든 비교 대상에 k번 이상의 비교셋을 구축하게 되며, 어떻게 비교셋을 구축하느냐가 학습에 영향을 끼치게 된다. 이 논문에서는 크라우드 소싱 기반의 딥러닝 선호도 측정을 위한 쌍체 비교 셋을 생성하는 새로운 알고리즘인 k-disjoint 비교셋 생성 알고리즘과 k-체이닝 비교셋 생성 알고리즘을 제안한다. 특히 k-체이닝 알고리즘은 기존의 원형 생성 알고리즘과 같이 데이터 간의 연결성을 보장하면서도 안정적인 선호도 평가를 지원할 수 있는 랜덤적 성격도 함께 가지고 있음을 실험에서 확인하였다.

Keywords

Acknowledgement

이 연구는 2020년도 정부(교육부)의 재원으로 한국연구재단의 기초연구사업(No.2020R1F1A1048432)과 2021년 한국국토정보공사 공간정보연구원의 산학협력 R&D 지원사업 자유과제 지원에 의하여 수행된 연구임.

References

  1. Burton, M. L. (2003). Too Many Questions? The Uses of Incomplete Cyclic Designs for Paired Comparisons. Field Methods. 15(2), 115-130. https://doi.org/10.1177/1525822X03015002001
  2. Chen, X., Bennett, P.N., Collins-Thompson, K. and Horvitz, E. (2013). Pairwise ranking aggregation in a crowdsourced setting. In proceedings of the 6th ACM International Conference on Web Search and Data Mining(WSDM), 193-202.
  3. Fisher, R. A., & Yates, F. (1953). Statistical tables for biological. Agricultural and Medical Research. Hafner Publishing Company, 26-27.
  4. Furnkranz, J., & Hullermeier, E. (2003). Pairwise preference learning and ranking. Lecture Notes in Artificial Intelligence (Subseries of Lecture Notes in Computer Science), 2837, 145-156.
  5. Jeong, Y. & Lee., C. (2021). Indoor autonomous driving through parallel reinforcement learning of virtual and real environments. Journal of the Korea Industrial Information Systems Research, 26(4), 11-18. https://doi.org/10.9723/JKSIIS.2021.26.4.011
  6. Kim, J. H., & Vu, V. H. (2003). Generating random regular graphs. Proceedings of the thirty-fifth annual ACM symposium on Theory of computing, STOC'03, 213-222.
  7. Koczkodaj, W. W., & Szybowski, J. (2015). Pairwise comparisons simplified. Applied Mathematics and Computation, 253, 387-394. https://doi.org/10.1016/j.amc.2014.12.069
  8. Knuth, D. E. (1969). Seminumerical algorithms. The Art of Computer Programming. Vol. 2. Reading, MA: Addison Wesley. 139-140.
  9. Lee, K., Nam, K., & Lee, C. (2022). A study on the walkability scores in Jeonju city using multiple regression model. Journal of the Korea Industrial Information Systems Research, 27(4), 1-10. https://doi.org/10.9723/JKSIIS.2022.27.4.001
  10. Nam, S., Lee, M., Heo, C., & Choi, K. (2020). Cost-effective multi-task crowdsourcing method for knowledge extraction. KIISE Transactions on Computing Practices, 26(11), 507-512. https://doi.org/10.5626/KTCP.2020.26.11.507
  11. Miranda, E., Bourque, P., & Abran, A. (2009). Sizing user stories using paired comparisons. Information and Software Technology, 51(9), 1327-1337. https://doi.org/10.1016/j.infsof.2009.04.003
  12. Saha, A., Shivanna, R. & Bhattacharyya, C. (2019). How many pairwise preferences do we need to rank a graph consistently?. 33rd AAAI Conference on Artificial Intelligence, AAAI 2019, 4830-4837.
  13. Sunahase, T., Baba, Y., & Kashima, H. (2017). Pairwise HITS: Quality estimation from pairwise comparisons in creator-evaluator crowdsourcing process. 31st AAAI Conference on Artificial Intelligence, AAAI 2017, Kleinberg, 977-983.
  14. Whang, S. E., Lofgren, P., & Garcia-Molina, H. (2013). Question selection for crowd entity resolution. Proceedings of the VLDB Endowment, 6(6), 349-360. https://doi.org/10.14778/2536336.2536337
  15. Wormald, N. C. (1999). Models of random regular graphs. London Mathematical Society Lecture Note Series, 239-298.
  16. Yoo, S. (2019). SPGS: Smart parking space guidance system based on user preferences in a parking lot. Journal of the Korea Industrial Information Systems Research, 24(4), 29-36. https://doi.org/10.9723/JKSIIS.2019.24.4.029