Browse > Article
http://dx.doi.org/10.22156/CS4SMB.2019.9.9.001

Survey on Out-Of-Domain Detection for Dialog Systems  

Jeong, Young-Seob (Department of Big Data Engineering, Soonchunhyang University)
Kim, Young-Min (Department of Big Data Engineering, Soonchunhyang University)
Publication Information
Journal of Convergence for Information Technology / v.9, no.9, 2019 , pp. 1-12 More about this Journal
Abstract
A dialog system becomes a new way of communication between human and computer. The dialog system takes human voice as an input, and gives a proper response in voice or perform an action. Although there are several well-known products of dialog system (e.g., Amazon Echo, Naver Wave), they commonly suffer from a problem of out-of-domain utterances. If it poorly detects out-of-domain utterances, then it will significantly harm the user satisfactory. There have been some studies aimed at solving this problem, but it is still necessary to study about this intensively. In this paper, we give an overview of the previous studies of out-of-domain detection in terms of three point of view: dataset, feature, and method. As there were relatively smaller studies of this topic due to the lack of datasets, we believe that the most important next research step is to construct and share a large dataset for dialog system, and thereafter try state-of-the-art techniques upon the dataset.
Keywords
Dialog system; User utterance; Out-of-domain detection; Natural language understanding; Text classification;
Citations & Related Records
Times Cited By KSCI : 1  (Citation Analysis)
연도 인용수 순위
1 Google. http://www.google.com
2 Yahoo. http://www.yahoo.com
3 M. S. Seigel. (2013). Confidence Estimation for Automatic Speech Recognition Hypotheses. Doctoral dissertation, St Edmund's College.
4 J. C. Chappelier, M. Rajman, R. Aragues. & A. Rozenknop. (1999). Lattice Parsing for Speech Recognition. Traitement Automatique du Langage Naturel, 95-104.
5 A. Graves. & N. Jaitly. (2014). Towards End-to-End Speech Recognition with Recurrent Neural Networks. Proceedings of the 31th International Conference on Machine Learning, 1764-1772.
6 H. Khouzaimi, R. Laroche & F. Lefevre. (2014). An Easy Method to Make Dialogue Systems Incremental. Proceedings of the 15th Annual Meeting of the Special Interest Group on Discourse and Dialogue, 98-107.
7 C. Shi, M. Verhagen & J. Pustejovsky. (2014). A Conceptual Framework of Online Natural Language Processing Pipeline Application. Proceedings of the Workshop on Open Infrastructures and Analysis Frameworks for HLT, 53-59.
8 X. Liu., R. Sarikaya., L. Zhao., Y. Ni. & Y. C. Pan. (2016). Personalized Natural Language Understanding. Proceedings of the 17th Annual Conference of the International Speech Communication Association, 1146-1150.
9 P. Xu. & R. Sarikaya. (2014). Contextual Domain Classification in Spoken Language Understanding Systems Using Recurrent Neural Network. Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, 136-140.
10 D. Wang., D. H. Tur & G. Tur. (2013). Understanding Computer-Directed Utterances in Multi-User Dialog Systems. Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, 8377-8381.
11 C. Lee., S. Jung., S. Kim & G. G. Lee. (2009). Example-Based Dialog Modeling for Practical Multi-Domain Dialog System. Speech Communication, 51, 466-484.   DOI
12 R. Meena. (2016). Data-Driven Methods for Spoken Dialogue Systems. Doctoral dissertation, KTH Royal Institute of Technology.
13 B. E. Boser, I. M. Guyon & V. N. Vapnik. (1992). A Training Algorithm For Optimal Margin Classifiers. Proceedings of the fifth Annual Workshop on Computational Learning Theory, 144-152.
14 M. Core. & J. Allen. (1997). Coding Dialogs With the DAMSL Annotation Scheme. Proceedings of the Working Notes of the AAAI Fall Symposium on Communicative Action in Humans and Machines.
15 G. Tur, A. Deoras & D. Hakkani-Tur. (2014). Detecting Out-Of-Domain Utterances Addressed to A Virtual Personal Assistant. Proceedings of the 15th Annual Conference of the International Speech Communication Association, 283-287.
16 E. Shriberg, A. Stolcke, D. Hakkani-Tur & L. Heck. (2012). Learning When to Listen: Detecting System-Addressed Speech in Human-Human-Computer Dialog. Proceedings of the 13th Annual Conference of the International Speech Communication Association, 334-337.
17 A. Stolcke et al. (2000). Dialogue Act Modeling For Automatic Tagging and Recognition of Conversational Speech. Computational Linguistics, 26(3), 339-373.   DOI
18 I. R. Lane. & T. Kawahara. (2005). Incorporating Dialogue Context and Topic Clustering in Out-of-Domain Detection. Proceedings of the International Conference on Acoustics, Speech, and Signal Processing, 1045-1048.
19 I. R. Lane., T. Kawahara., T. Matsui. & S. Nakamura. (2004). Out-Of-Domain Detection Based On Confidence Measures From Multiple Topic Classification. Proceedings of the International Conference on Acoustics, Speech, and Signal Processing, 757-760.
20 I. R. Lane., T. Kawahara., T. Matsui. & S. Nakamura. (2004). Topic Classification and Verification Modeling For Out-Of-Domain Utterance Detection. Proceedings of the 8th International Conference on Spoken Language Processing.
21 X. Zhang., J. Zhao. & Y. Lecun. (2015). Character-Level Convolutional Networks for Text Classification. Proceedings of the 28th International Conference on Neural Information Processing Systems, 649-657.
22 Y. Fujita. S. Takeuchi, H. Kawanami, T. Matsui, H. Saruwatari & K. Shikano. (2011). Out-of-Task Utterance Detection Based on Bag-of-Words Using Automatic Speech Recognition Results. Proceedings of the third Annual Summit and Conference of Asia-Pacific Signal and Information Processing Association.
23 Y. S. Jeong. (2017). Experimental Analysis for Out-Of-Domain Detection Using Features of Word Positions in Sentence. Proceedings of the Spring Conference of Korean Society for Internet Information, 18(1).
24 Wordnet. https://wordnet.princeton.edu/
25 D. Hogan, J. Leveling, H. Wang, P. Ferguson & C. Gurrin. (2013). SMS Normalisation, Retrieval and Out-of-Domain Detection Approaches for SMS-Based FAQ Retrieval. Multilingual Information Access in South Asian Languages, 184-196.
26 S. Ryu, D. Lee, G. G. Lee, K. Kim & H. Noh. (2014). Exploiting Out-Of-Vocabulary Words For Out-Of-Domain Detection in Dialog Systems. Proceedings of the International Conference on Big Data and Smart Computing, 165-168.
27 D. A. Freedman. (2009). Statistical Models: Theory and Practice. Cambridge University Press.
28 M. Nakano, S. Sato, K. Komatani, K. Matsuyama, K. Funakoshi & H. G. Okuno. (2011). A Two-Stage Domain Selection Framework for Extensible Multi-Domain Spoken Dialogue Systems. Proceedings of the 12th Annual Meeting of the Special Interest Group on Discourse and Dialogue, 18-29.
29 Springer. (2003). The Elements of Statistical Learning. Berlin: T. Hastie., R. Tibshirani. & J. Friedman.
30 S. Deerwester, S. T. Dumais, G. W. Furnas, T. K. Landauer & R. Harshman. (1990). Indexing by Latent Semantic Analysis. Journal of the American Society for Information Science, 41(6), 391-407.   DOI
31 A. L. Berger, S. A. D. Pietra & V. J. D. Pietra. (1996). A Maximum Entropy Approach to Natural Language Processing. Computational Linguistics, 22(1), 39-71.
32 S. Ryu, S. Kim, J. Choi, H. Yu & G. G. Lee. (2017). Neural Sentence Embedding Using Only In-Domain Sentences for Out-Of-Domain Sentence Detection in Dialog Systems. Pattern Recognition Letter, 88, 26-32.   DOI
33 S. Hochreiter. & J. Schmidhuber. (1997). Long Short-Term Memory. Neural Computation, 9(8), 1735-1780.   DOI
34 J. K. Kim. & Y. B. Kim. (2018). Joint Learning of Domain Classification and Out-of-Domain Detection with Dynamic Class Weighting for Satisficing False Acceptance Rates. Proceedings of 19th Annual Conference of the International Speech Communication Association, 556-560.
35 Y. S. Jeong. (2018). Out-Of-Domain Detection Using Hierarchical Dirichlet Process. Journal of The Korea Society of Computer and Information, 23(1), 17-24.   DOI