Browse > Article
http://dx.doi.org/10.9723/jksiis.2022.27.5.001

Generating Pairwise Comparison Set for Crowed Sourcing based Deep Learning  

Yoo, Kihyun (군산대학교 컴퓨터정보통신공학부)
Lee, Donggi (군산대학교 컴퓨터정보통신공학부)
Lee, Chang Woo (군산대학교 컴퓨터정보통신공학부)
Nam, Kwang Woo (군산대학교 컴퓨터정보통신공학부)
Publication Information
Journal of Korea Society of Industrial Information Systems / v.27, no.5, 2022 , pp. 1-11 More about this Journal
Abstract
With the development of deep learning technology, various research and development are underway to estimate preference rankings through learning, and it is used in various fields such as web search, gene classification, recommendation system, and image search. Approximation algorithms are used to estimate deep learning-based preference ranking, which builds more than k comparison sets on all comparison targets to ensure proper accuracy, and how to build comparison sets affects learning. In this paper, we propose a k-disjoint comparison set generation algorithm and a k-chain comparison set generation algorithm, a novel algorithm for generating paired comparison sets for crowd-sourcing-based deep learning affinity measurements. In particular, the experiment confirmed that the k-chaining algorithm, like the conventional circular generation algorithm, also has a random nature that can support stable preference evaluation while ensuring connectivity between data.
Keywords
Preference prediction; pairwise comparision; k-regular graph;
Citations & Related Records
연도 인용수 순위
  • Reference
1 Nam, S., Lee, M., Heo, C., & Choi, K. (2020). Cost-effective multi-task crowdsourcing method for knowledge extraction. KIISE Transactions on Computing Practices, 26(11), 507-512.   DOI
2 Chen, X., Bennett, P.N., Collins-Thompson, K. and Horvitz, E. (2013). Pairwise ranking aggregation in a crowdsourced setting. In proceedings of the 6th ACM International Conference on Web Search and Data Mining(WSDM), 193-202.
3 Fisher, R. A., & Yates, F. (1953). Statistical tables for biological. Agricultural and Medical Research. Hafner Publishing Company, 26-27.
4 Wormald, N. C. (1999). Models of random regular graphs. London Mathematical Society Lecture Note Series, 239-298.
5 Sunahase, T., Baba, Y., & Kashima, H. (2017). Pairwise HITS: Quality estimation from pairwise comparisons in creator-evaluator crowdsourcing process. 31st AAAI Conference on Artificial Intelligence, AAAI 2017, Kleinberg, 977-983.
6 Yoo, S. (2019). SPGS: Smart parking space guidance system based on user preferences in a parking lot. Journal of the Korea Industrial Information Systems Research, 24(4), 29-36.   DOI
7 Burton, M. L. (2003). Too Many Questions? The Uses of Incomplete Cyclic Designs for Paired Comparisons. Field Methods. 15(2), 115-130.   DOI
8 Furnkranz, J., & Hullermeier, E. (2003). Pairwise preference learning and ranking. Lecture Notes in Artificial Intelligence (Subseries of Lecture Notes in Computer Science), 2837, 145-156.
9 Koczkodaj, W. W., & Szybowski, J. (2015). Pairwise comparisons simplified. Applied Mathematics and Computation, 253, 387-394.   DOI
10 Lee, K., Nam, K., & Lee, C. (2022). A study on the walkability scores in Jeonju city using multiple regression model. Journal of the Korea Industrial Information Systems Research, 27(4), 1-10.   DOI
11 Miranda, E., Bourque, P., & Abran, A. (2009). Sizing user stories using paired comparisons. Information and Software Technology, 51(9), 1327-1337.   DOI
12 Saha, A., Shivanna, R. & Bhattacharyya, C. (2019). How many pairwise preferences do we need to rank a graph consistently?. 33rd AAAI Conference on Artificial Intelligence, AAAI 2019, 4830-4837.
13 Jeong, Y. & Lee., C. (2021). Indoor autonomous driving through parallel reinforcement learning of virtual and real environments. Journal of the Korea Industrial Information Systems Research, 26(4), 11-18.   DOI
14 Kim, J. H., & Vu, V. H. (2003). Generating random regular graphs. Proceedings of the thirty-fifth annual ACM symposium on Theory of computing, STOC'03, 213-222.
15 Knuth, D. E. (1969). Seminumerical algorithms. The Art of Computer Programming. Vol. 2. Reading, MA: Addison Wesley. 139-140.
16 Whang, S. E., Lofgren, P., & Garcia-Molina, H. (2013). Question selection for crowd entity resolution. Proceedings of the VLDB Endowment, 6(6), 349-360.   DOI