Browse > Article

Analysis of Novelty Detection Properties of Autoassociative MLP  

Lee, Hyoung-joo (Department of Industrial Engineering, Seoul National University)
Hwang, Byung-ho (Digital Media Research Lab, LG Electronics)
Cho, Sungzoon (Department of Industrial Engineering, Seoul National University)
Publication Information
Journal of Korean Institute of Industrial Engineers / v.28, no.2, 2002 , pp. 147-161 More about this Journal
Abstract
In novelty detection, one attempts to discriminate abnormal patterns from normal ones. Novelty detection is quite difficult since, unlike usual two class classification problems, only normal patterns are available for training. Auto-Associative Multi-Layer Perceptron (AAMLP) has been shown to provide a good performance based upon the property that novel patterns usually have larger auto-associative errors. In this paper, we give a mathematical analysis of 2-layer AAMLP's output characteristics and empirical results of 2-layer and 4-layer AAMLPs. Various activation functions such as linear, saturated linear and sigmoid are compared. The 2-layer AAMLPs cannot identify non-linear boundaries while the 4-layer ones can. When the data distribution is multi-modal, then an ensemble of AAMLPs, each of which is trained with pre-clustered data is required. This paper contributes to understanding of AAMLP networks and leads to practical recommendations regarding its use.
Keywords
novelty detection; auto-associative multi-layer perceptron; principal component analysis; non-linear boundary;
Citations & Related Records
연도 인용수 순위
  • Reference
1 Bishop, C. M. (1994), Novelty Detection and Neural Network Validation, lEE Proceedings of Visual Image Signal Process, 141(4),217-222   DOI   ScienceOn
2 Cho, S., Han, C., Han, D. and Kim, H. (2000a), Web Based Keystroke Dynamics Identity Verification using Neural Networks, Journal of Organizational Computing and Electronic Commerce. 10(4),295-307
3 Japkowicz, N., Myers, C. and Gluck, M. (1995), A Novelty Detection Approach to Classification, Proceedings of the Fourteenth International Conference on Artificial Intelligence, 518-523
4 Kramer, M. (1991), Nonlinear Principal Component Analysis using Autoassociative Neural Networks, AIChE Journal, 37(2), 233-243   DOI
5 Obaidat, M. S. and Sadoun, B. (1997), Verification of Computer Users using Keystroke Dynamics, IEEE Transactions on Systems, Man, and Cybernetics, 27(2),261-269
6 Reddy, V., Riley, P. and Mavrovouniotis, M. (1996), Analysis of Plant Measurements through Input-training Neural Networks, Computers Chemical Engineering, 20., Suppl., 889-894
7 Cho, S. and Han, D. (2000b), Apparatus for Authenticating an Individual Based on a Typing Pattern by Using a Neural Network System, Patent No. 6,151,593, US Patent and Trademark Office, Washington DC 20231, USA, Granted on Nov. 21, 2000
8 Frosini, A., Gori, M. and Priami, P. (1996), A Neural Network based Model for Paper Currency Recognition and Verification, IEEE Transactions on Neural Networks, 7(6),1482-1490   DOI   ScienceOn
9 Xu, L. and Jordan, M. I. (1993), Unsupervised Learning by EM Algorithm based on Finite Mixture of Gaussians, Proceedings of World Congress of Neural Networks, 431-434
10 Kaski, S. and Lagus, K. (1996), Comparing Self-organizing Maps, Proceedings of International Conference on Artificial Neural Networks,809-814
11 Ikbal, M., Misra, H. and Yegnanarayana, B. (1999), Analysis of Autoassociative Mapping Neural Networks, Proceedings of International Joint Conference on Neural Networks, #854
12 Yin, H. and Allinson, N. M. (2001), Self-organizing Mixture Networks for Probability Density Estimation, IEEE Transactions on Neural Networks, 12(2), 405-411   DOI   ScienceOn
13 Malthouse, E. C. (1998), Limitations of Non-linear PCA as Performed with Generic Neural Networks, IEEE Transactions on Neural Networks, 9(1), 165-173   DOI   ScienceOn
14 Baldi, P. and Hornik, K. (1989), Neural Networks and Principal Component Analysis: Learning from Examples without Local Minima, Neural Networks, 2, 53-58
15 Bianchini, M., Frasconi, P. and Gori, M. (1995), Learning in Multi-layered Networks used as Autoassociators, IEEE Transactions on Neural Networks, 6(2),512-515   DOI   ScienceOn