Browse > Article
http://dx.doi.org/10.9708/jksci.2018.23.01.017

Out-Of-Domain Detection Using Hierarchical Dirichlet Process  

Jeong, Young-Seob (Dept. of Big Data Engineering, Soonchunhyang University)
Abstract
With improvement of speech recognition and natural language processing, dialog systems are recently adapted to various service domains. It became possible to get desirable services by conversation through the dialog system, but it is still necessary to improve separate modules, such as domain detection, intention detection, named entity recognition, and out-of-domain detection, in order to achieve stable service offer. When it misclassifies an in-domain sentence of conversation as out-of-domain, it will result in poor customer satisfaction and finally lost business. As there have been relatively small number of studies related to the out-of-domain detection, in this paper, we introduce a new method using a hierarchical Dirichlet process and demonstrate the effectiveness of it by experimental results on Korean dataset.
Keywords
Out-of-domain detection; Dialog system; Topic modeling; Hierarchical Dirichlet process;
Citations & Related Records
연도 인용수 순위
  • Reference
1 Amazon Echo, https://www.amazon.com
2 SK Telecom Nugu, http://www.nugu.co.kr/main
3 Naver WAVE, https://clova.ai/ko
4 Naver AWAY IVI platform, http://www.naverlabs.com/newsroom/IVI.html
5 Hyundai-card Buddy, www.hyundaicard.com/
6 Lidia Mangu and Eric Brill, "Lattice Compression in the Consensual Post-Processing Framework," in Proceedings of the Third World Multiconference on Systemics, Cybernetics and Informatics, pp. 246-252, 2000.
7 Ian R. Lane, Tatsuya Kawahara, Tomoko Matsui, and Satoshi Nakamura, "Out-Of-Domain Detection Based on Confidence Measures From Multiple Topic Classificati on," in Proceedings of the International Conference on Acoustics, Speech, and Signal Processing, pp. 757-760, 2004.
8 Seonghan Ryu and Donghyeon Lee, Gary Geunbae Lee, Kyungduk Kim, and Hyungjong Noh, "Exploiting Out-Of-Vocabulary Words For Out-Of-Domain Detection in Dialog Systems," in Proceedings of the International Conference on Big Data and Smart Computing, pp. 165-168, 2014.
9 Deirdre Hogan, Johannes Leveling, Hongyi Wang, Paul Ferguson, and Cathal Gurrin, "SMS Normalisation, Retrieval and Out-of-Domain Detection Approaches for SMS-Based FAQ Retrieval," Multilingual Information Access in South Asian Languages, pp. 184-196, 2013.
10 Mikio Nakano, Shun Sato, Kazunori Komatani, Kyoko Matsuyama, Kotaro Funakoshi, and Hiroshi G. Okuno, "A Two-Stage Domain Selection Framework for Extensible Multi-Domain Spoken Dialogue Systems," in Proceedings of the 12th Annual Meeting of the Special Interest Group on Discourse and Dialogue, Portland, Oregon, USA, pp. 18-29, June 17-18, 2011.
11 Yoko Fujita, Shota Takeuchi, Hiromichi Kawanami, Tomoko Matsui, Hiroshi Saruwatari, and Kiyohiro Shikano, "Out-of-Task Utterance Detection Based on Bag-of-Words Using Automatic Speech Recognition Results," in Proceedings of the third Annual Summit and Conference of Asia-Pacific Signal and Information Processing Association, Xi'an, China, October 18-21, 2011.
12 Wordnet, https://wordnet.princeton.edu/
13 Naver search, https://www.naver.com
14 David A. Freedman, "Statistical Models: Theory and Practice," Cambridge University Press, 2009.
15 Google search, https://www.google.com
16 Young-Seob Jeong, "Experimental Analysis for Out-Of-Domain Detection Using Features of Word Positions in Sentences," in Proceedings of the Spring Conference of Korean Society for Internet Information, April 21-22, 2017.
17 Trevor Hastie, Robert Tibshirani, and Jerome Friedman, "The elements of statistical learning," Springer, New York, 2003.
18 Scott Deerwester, Susan T. Dumais, George W. Furnas, Thomas K. Landauer, and Richard Harshman, "Indexing by Latent Semantic Analysis," Journal of the American Society for Information Science (JASIS), Vol. 41, No. 6, pp. 391-407, 1990.   DOI
19 Bernhard E. Boser, Isabelle M. Guyon, and Vladimir N. Vapnik, "A Training Algorithm For Optimal Margin Classifiers," in Proceedings of the fifth Annual Workshop on Computational Learning Theory, Pittsburgh, PA, USA, pp. 144-152, 1992.
20 Ian R. Lane, Tatsuya Kawahara, and Shinichi Ueno, "Example-Based Training of Dialogue Planning Incorporating User and Situation Models," in Proceedings of the 8th International Conference on Spoken Language Processing, Jeju, Korea, October 4-8, 2004.
21 Adam L. Berger, Stephen A. Della Pietra, and Vincent J. Della Pietra, "A Maximum Entropy Approach to Natural Language Processing," Computational Linguistics, Vol. 22, No. 1, pp. 39-71, 1996.
22 Seonghan Ryu, Seokhwan Kim, Junhwi Choi, Hwanjo Yu, and Gary Geunbae Lee, "Neural Sentence Embedding Using Only In-Domain Sentences For Out-Of-Domain Sentence Detection in Dialog Systems," Pattern Recognition Letters, Vol. 88, pp. 26-32, 2017.   DOI
23 Leo Breiman, "Random Forests," Machine Learning, Vol. 45, No. 1, pp. 5-32, 2001.   DOI
24 Yee Whye Teh, Michael I. Jordan, Matthew J. Beal, and David M. Blei, "Hierarchical Dirichlet Processes," Journal of the American Statistical Association, Vol. 101, No. 476, pp. 1566-1581, 2006.   DOI
25 David M. Blei, Andrew Y. Ng, and Michael I. Jordan, "Latent Dirichlet Allocation," Journal of Machine Learning Research, 3, pp. 993-1022, January, 2003.
26 Machal Rosen-Zvi, Chaitanya Chemudugunta, Thomas Griffiths, Padhraic Smyth, and Mark Steyvers, "Learning Author-Topic Models," ACM Transactions on Information Systems, Vol. 28, No. 1, pp. 1-38, 2010.
27 Young-Seob Jeong and Ho-Jin Choi, "Sequential Entity Group Topic Model for Getting Topic Flows of Entity Groups Within One Document," in Proceedings of the 16th Pacific-Asia Conference on Knowledge Discovery and Data Mining, Kuala Lumpur, Malaysia, pp. 366-378, 29 May - 1 June, 2012.
28 Thomas L. Griffiths and Mark Steyvers, "Finding Scientific Topics," in Proceedings of the National Academy of Sciences of the United States of America, pp. 5228-5235, 2004.
29 Ian R. Lane and Tatsuya Kawahara, "Incorporating Dialogue Context and Topic Clustering in Out-of-Domain Detection," in Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, Philadelphia, Pennsylvania, USA, pp. 1045-1048, March 18-23, 2005.
30 John C. Platt, "Sequential Minimal Optimization: A Fast Algorithm for Training Support Vector Machines," Advances in Kernel Methods - Support Vector Learning, MIT Press, 1998.
31 Sepp Hochreiter and Jurgen Schmidhuber, "Long Short-term Memory," Neural Computation, Vol. 9, No. 8, pp. 1735-1780, 1997.   DOI