ETRI Journal
- Volume 28 Issue 3
- /
- Pages.337-346
- /
- 2006
- /
- 1225-6463(pISSN)
- /
- 2233-7326(eISSN)
Dependency Structure Applied to Language Modeling for Information Retrieval
- Lee, Chang-Ki (Embedded Software Research Division, ETRI) ;
- Lee, Gary Geun-Bae (Department of Computer Science & Engineering, POSTECH) ;
- Jang, Myung-Gil (Embedded Software Research Division, ETRI)
- Received : 2005.01.31
- Published : 2006.06.30
Abstract
In this paper, we propose a new language model, namely, a dependency structure language model, for information retrieval to compensate for the weaknesses of unigram and bigram language models. The dependency structure language model is based on the first-order dependency model and the dependency parse tree generated by a linguistic parser. So, long-distance dependencies can be naturally captured by the dependency structure language model. We carried out extensive experiments to verify the proposed model, where the dependency structure model gives a better performance than recently proposed language models and the Okapi BM25 method, and the dependency structure is more effective than unigram and bigram in language modeling for information retrieval.