Browse > Article

An MILP Approach to a Nonlinear Pattern Classification of Data  

Kim, Kwangsoo (Department of Industrial Systems and Information Engineering, Korea University)
Ryoo, Hong Seo (Department of Industrial Systems and Information Engineering, Korea University)
Publication Information
Journal of Korean Institute of Industrial Engineers / v.32, no.2, 2006 , pp. 74-81 More about this Journal
Abstract
In this paper, we deal with the separation of data by concurrently determined, piecewise nonlinear discriminant functions. Toward the end, we develop a new $l_1$-distance norm error metric and cast the problem as a mixed 0-1 integer and linear programming (MILP) model. Given a finite number of discriminant functions as an input, the proposed model considers the synergy as well as the individual role of the functions involved and implements a simplest nonlinear decision surface that best separates the data on hand. Hence, exploiting powerful MILP solvers, the model efficiently analyzes any given data set for its piecewise nonlinear separability. The classification of four sets of artificial data demonstrates the aforementioned strength of the proposed model. Classification results on five machine learning benchmark databases prove that the data separation via the proposed MILP model is an effective supervised learning methodology that compares quite favorably to well-established learning methodologies.
Keywords
machine learning; data mining; mixed integer and linear programming; classification;
Citations & Related Records
연도 인용수 순위
  • Reference
1 Mangasarian, O. L. (2000), Generalized support vector machines, In A. Smola, P. Bartlett, B. Scholkopf and D. Schuurmans, editors, Advances in Large margin classifiers, MIT Press, 135-146
2 Wolberg, W. H. and Mangasarian, O. L. (1990), Multisurface method of pattern separation for medical diagnosis applied to breast cytology, Proceedings of the National Academy of Sciences, 87, 9193-9196
3 Bennett, K. P. and Mangasarian, O. L. (1994), Bilinear separation of two sets in n-space, Computational Optimization and Applications, 2, 207-227   DOI
4 Osuna, E., Freund, R., and Girosi, F. (1997), Training support vector machines: an application to face detection, In IEEE Conference on Computer Vision and Pattern Recognition, Puerto Rico, 130-136
5 Boros, E., Hammer, P. L., Ibaraki, T., Kogan, A., Mayoraz, E., and Muchnik, I. (2000), An implementation of logical analysis of data, IEEE Transactions on Knowledge and Data Engineering, 12(2), 292-306   DOI   ScienceOn
6 Carter, C. and Catlett, J. (1997), Assessing credit card applications using machine learning, IEEE Expert, Fall, 71-79
7 Mangasarian, O. L. and Musicant, D. R. (2000), Data discrimination via nonlinear generalized support vector machines, In M. C. Ferris, O. L. Mangasarian, and J.-S. Pang, editors, Complementarity: Applications, Algorithms and Extensions, chapter 1. Kluwer Academic Publishers
8 Ryoo, H. S. and Sahinidis, N. V. (2001), Analysis of bounds for multilinear functions, Journal of Global Optimization, 19(4), 403-424   DOI   ScienceOn
9 Ullman, J. R. (1973), Pattern recognition techniques, Crane, London
10 ILOG CPLEX 9.0 User's Manual, ILOG CPLEX Division, Incline, Nevada, October 2001
11 Murphy, P. M. and Aha, D. W. (1994), UCI repository of machine learning databases, Department of Computer Science, University of California at Irvine, CA
12 Bradley, P. S. and Mangasarian, O. L. (2000), Massive data discrimination via linear support vector machines, Optimization Methods and Software, 13, 1-10   DOI
13 Falk, J. E. and Lopez-Cardona, E. (1997), The surgical separation of sets, Journal of Global Optimization, 11, 433-462   DOI   ScienceOn
14 Mangasarian, O. L. (1968), Multisurface method of pattern separation, IEEE Transactions on Information Theory, 14(6), 801-807   DOI
15 Mangasarian, O. L. (1965), Linear and nonlinear separation of patterns by linear programming, Operations Research, 13, 444-452   DOI   ScienceOn
16 Nakayama, H. and Kagaku, N. (1998), Pattern classification by linear goal programming and its extensions, Journal of Global Optimization, 12, 111-126   DOI   ScienceOn
17 Bennett, K. P. and Mangasarian, O. L. (1992), Robust linear programming discrimination of two linearly inseparable sets, Optimization Methods and Software, 1, 23-34   DOI   ScienceOn
18 Mangasarian, O. L. (1993), Mathematical programming in neural networks, ORSA Journal on Computing, 5, 349-360   DOI   ScienceOn
19 Ryoo, H. S. and Jang, I.-Y. (2005), MILP approach to pattern generation in logical analysis of data, Machine Learning, submitted
20 Ryoo, H. S. and Sahinidis, N. V. (2003), Global optimization of multiplicative programs, Journal of Global Optimization, 26, 387-418   DOI   ScienceOn
21 Bennett, K. P. (1993), Decision tree construction via linear programming, In Proceedings of the 9th Conference on Artificial Intelligence for Applications, Orlando, Florida, 212-218
22 Frank, M. and Wolfe, P. (1956), An algorithm for quadratic programming, Naval Research Logistics Quarterly, 3, 95-110   DOI
23 Megiddo, N. (1988), On the complexity of polyhedral separability, Discrete and Computational Geometry, 3, 325-337   DOI
24 Al-Khayyal, F. A. and Falk, J. E. (1983), Jointly constrained biconvex programming, Mathematics of Operations Research, 8(2), 273-286   DOI   ScienceOn
25 Mangasarian, O. L., Setiono, R., and Wolberg, W. H. (1990), Pattern recognition via linear programming : Theory and application to medical diagnosis, Large-Scale Numerical Optimization, 22-31
26 Mangasarian, O. L., Street, W. N., and Wolberg, W. H. (1995), Breast cancer diagnosis and prognosis via linear programming, Operations Research, 43(4), 570-577   DOI   ScienceOn