Browse > Article

(Resolving Prepositional Phrase Attachment and POS Tagging Ambiguities using a Maximum Entropy Boosting Model)  

박성배 (서울대학교 컴퓨터신기술공동연구소)
Publication Information
Journal of KIISE:Software and Applications / v.30, no.5_6, 2003 , pp. 570-578 More about this Journal
Abstract
Maximum entropy models are promising candidates for natural language modeling. However, there are two major hurdles in applying maximum entropy models to real-life language problems, such as prepositional phrase attachment: feature selection and high computational complexity. In this paper, we propose a maximum entropy boosting model to overcome these limitations and the problem of imbalanced data in natural language resources, and apply it to prepositional phrase (PP) attachment and part-of-speech (POS) tagging. According to the experimental results on Wall Street Journal corpus, the model shows 84.3% of accuracy for PP attachment and 96.78% of accuracy for POS tagging that are close to the state-of-the-art performance of these tasks only with small efforts of modeling.
Keywords
Maximum Entropy Model; PP Attachment; POS Tagging; Boosting; Active learning;
Citations & Related Records
연도 인용수 순위
  • Reference
1 A. Ratnaparkhi, J. Reynar, and S. Roukos, 'A maximum entropy model for prepositional phrase attachment,' In Proceedings of the Human Language Technology Workshop, pp. 250-255, 1994   DOI
2 M. Collins and J. Brooks, 'Prepositional phrase attachment through a backed-off model,' In Proceedings of the Third Workshop on Very Large Corpora, pp. 27-38, 1995
3 E. Brill and P. Resnik, 'A rule-based approach to prepositional phrase attachment disambiguation,' In Proceedings of the 15th International Conference on Computational Linguistics, pp. 1198-1204, 1994
4 R. Weischedel, M. Meteer, R. Schwartz, L. Ramshaw and J. Palmucci, 'Coping with ambiguity and unknown words through probabilistic models,' Computational Linguistics, Vol. 19, No. 2, pp. 359-382, 1994
5 E. Brill, 'Some advances in transformation-based part of speech tagging,' In Proceedings of the 12th National Conference on Artificial Intelligence, pp. 722-727, 1994
6 H. Schmid, 'Part-of-speech tagging with neural networks,' In Proceedings of the 15th International Conference on Computational Linguistics, pp. 172-176, 1994
7 T. Cover and J. Thomas, Element of information theory, John Wiley, 1991
8 A. Ratnaparkhi, Maximum Entropy Models for Natural Language Ambiguity Resolution, Ph.D thesis, University of Pennsylvannia, 1998
9 Y. Freund and R. Schapire, 'Experiments with a new boosting algorithm,' In Proceedings of the 13th International Conference on Machine Learning, pp. 148-156, 1996
10 M. Kubat and S. Matwin, 'Addressing the curse of imbalanced training sets-' One-sided selection,' In Proceedings of the 14th International Conference on Machine Learning, pp. 179-186, 1997
11 A. Ratnaparkhi, 'A maximum entropy model for part-of-speech tagging,' In Proceedings of the Empirical Methods in Natural Language Processing, pp. 133-142, 1996
12 R. Quinlan, C4.5:Programs for Machine Learning, Morgan Kaufmann Publishers, 1993
13 M. Kubat and S. Matwin, 'Addressing the curse of imbalanced training sets: one-sided selection,' In Proceedings of the 14th International Conference on Machine Learning, pp. 179-186, 1997
14 H. Baayen adn R. Sproat, 'Estimating lexical priors for low-frequency morphologically ambiguous forms,' Computational Linguistics, Vol. 22, No. 2, pp. 155-166, 1996
15 D. Darroch and D. Ratcliff, 'Generalized iterative scaling for log-linear models,' The Annals of Mathenntical Statistics, Vol. 43, No. 5, pp 1470-1480, 1972   DOI   ScienceOn
16 J. Steina and M. Nagao, 'Corpus based PP attachment ambiguity resolution with a semantic dictionary,' In Proceedings of the Fifth Workshop on Very Large Corpora, pp. 66-80, 1997
17 P. Pantel and D. Lin, 'An supervised approach to prepositional phrase attachment using con-textually similar words,' In Proceedings of the 38th Annual Meeting of the Association for Computational Linguistics, pp. 101-108, 2000
18 S. Katz, 'Estimation of probabilities from sparse data for the language model component of a speech recognizer,' IEEE Transactions on Acoustics, Speech and Signal Processing, Vol. 35, No. 3, pp. 400-401, 1987   DOI
19 S. Chen and J. Goodman, 'An empirical study of smoothing techniques for language modeling,' In Proceedings of the 34th Annual Meeting of the Association for Computational Linguistics, pp. 310-318, 1996   DOI
20 S. Abney, R. Schapire, and Y. Singer, 'Boosting applied to tagging and PP-attachment,' In Proceedings of the Joint SIGDAT Conference on Empirical Methods in Natural Language Processing and Very Large Corpora, pp. 38-45, 1999