Journal of Korean Institute of Industrial Engineers (대한산업공학회지)

Korean Institute of Industrial Engineers (대한산업공학회)
권호 표지

An MILP Approach to a Nonlinear Pattern Classification of Data

혼합정수 선형계획법 기반의 비선형 패턴 분류 기법

  • Kim, Kwangsoo (Department of Industrial Systems and Information Engineering, Korea University) ;
  • Ryoo, Hong Seo (Department of Industrial Systems and Information Engineering, Korea University)
  • 김광수 (고려대학교 산업시스템정보공학과) ;
  • 류홍서 (고려대학교 산업시스템정보공학과)
  • Published : 2006.06.30
Download PDF
⟨ Previous Next ⟩

Abstract

In this paper, we deal with the separation of data by concurrently determined, piecewise nonlinear discriminant functions. Toward the end, we develop a new $l_1$-distance norm error metric and cast the problem as a mixed 0-1 integer and linear programming (MILP) model. Given a finite number of discriminant functions as an input, the proposed model considers the synergy as well as the individual role of the functions involved and implements a simplest nonlinear decision surface that best separates the data on hand. Hence, exploiting powerful MILP solvers, the model efficiently analyzes any given data set for its piecewise nonlinear separability. The classification of four sets of artificial data demonstrates the aforementioned strength of the proposed model. Classification results on five machine learning benchmark databases prove that the data separation via the proposed MILP model is an effective supervised learning methodology that compares quite favorably to well-established learning methodologies.

Keywords

References

  1. Al-Khayyal, F. A. and Falk, J. E. (1983), Jointly constrained biconvex programming, Mathematics of Operations Research, 8(2), 273-286 https://doi.org/10.1287/moor.8.2.273
  2. Bennett, K. P. (1993), Decision tree construction via linear programming, In Proceedings of the 9th Conference on Artificial Intelligence for Applications, Orlando, Florida, 212-218
  3. Bennett, K. P. and Mangasarian, O. L. (1992), Robust linear programming discrimination of two linearly inseparable sets, Optimization Methods and Software, 1, 23-34 https://doi.org/10.1080/10556789208805504
  4. Bennett, K. P. and Mangasarian, O. L. (1994), Bilinear separation of two sets in n-space, Computational Optimization and Applications, 2, 207-227 https://doi.org/10.1007/BF01299449
  5. Boros, E., Hammer, P. L., Ibaraki, T., Kogan, A., Mayoraz, E., and Muchnik, I. (2000), An implementation of logical analysis of data, IEEE Transactions on Knowledge and Data Engineering, 12(2), 292-306 https://doi.org/10.1109/69.842268
  6. Bradley, P. S. and Mangasarian, O. L. (2000), Massive data discrimination via linear support vector machines, Optimization Methods and Software, 13, 1-10 https://doi.org/10.1080/10556780008805771
  7. Carter, C. and Catlett, J. (1997), Assessing credit card applications using machine learning, IEEE Expert, Fall, 71-79
  8. Falk, J. E. and Lopez-Cardona, E. (1997), The surgical separation of sets, Journal of Global Optimization, 11, 433-462 https://doi.org/10.1023/A:1008284015704
  9. Frank, M. and Wolfe, P. (1956), An algorithm for quadratic programming, Naval Research Logistics Quarterly, 3, 95-110 https://doi.org/10.1002/nav.3800030109
  10. ILOG CPLEX 9.0 User's Manual, ILOG CPLEX Division, Incline, Nevada, October 2001
  11. Mangasarian, O. L. (1965), Linear and nonlinear separation of patterns by linear programming, Operations Research, 13, 444-452 https://doi.org/10.1287/opre.13.3.444
  12. Mangasarian, O. L. (1968), Multisurface method of pattern separation, IEEE Transactions on Information Theory, 14(6), 801-807 https://doi.org/10.1109/TIT.1968.1054229
  13. Mangasarian, O. L. (1993), Mathematical programming in neural networks, ORSA Journal on Computing, 5, 349-360 https://doi.org/10.1287/ijoc.5.4.349
  14. Mangasarian, O. L. (2000), Generalized support vector machines, In A. Smola, P. Bartlett, B. Scholkopf and D. Schuurmans, editors, Advances in Large margin classifiers, MIT Press, 135-146
  15. Mangasarian, O. L. and Musicant, D. R. (2000), Data discrimination via nonlinear generalized support vector machines, In M. C. Ferris, O. L. Mangasarian, and J.-S. Pang, editors, Complementarity: Applications, Algorithms and Extensions, chapter 1. Kluwer Academic Publishers
  16. Mangasarian, O. L., Setiono, R., and Wolberg, W. H. (1990), Pattern recognition via linear programming : Theory and application to medical diagnosis, Large-Scale Numerical Optimization, 22-31
  17. Mangasarian, O. L., Street, W. N., and Wolberg, W. H. (1995), Breast cancer diagnosis and prognosis via linear programming, Operations Research, 43(4), 570-577 https://doi.org/10.1287/opre.43.4.570
  18. Megiddo, N. (1988), On the complexity of polyhedral separability, Discrete and Computational Geometry, 3, 325-337 https://doi.org/10.1007/BF02187916
  19. Murphy, P. M. and Aha, D. W. (1994), UCI repository of machine learning databases, Department of Computer Science, University of California at Irvine, CA
  20. Nakayama, H. and Kagaku, N. (1998), Pattern classification by linear goal programming and its extensions, Journal of Global Optimization, 12, 111-126 https://doi.org/10.1023/A:1008244409770
  21. Osuna, E., Freund, R., and Girosi, F. (1997), Training support vector machines: an application to face detection, In IEEE Conference on Computer Vision and Pattern Recognition, Puerto Rico, 130-136
  22. Ryoo, H. S. and Jang, I.-Y. (2005), MILP approach to pattern generation in logical analysis of data, Machine Learning, submitted
  23. Ryoo, H. S. and Sahinidis, N. V. (2001), Analysis of bounds for multilinear functions, Journal of Global Optimization, 19(4), 403-424 https://doi.org/10.1023/A:1011295715398
  24. Ryoo, H. S. and Sahinidis, N. V. (2003), Global optimization of multiplicative programs, Journal of Global Optimization, 26, 387-418 https://doi.org/10.1023/A:1024700901538
  25. Ullman, J. R. (1973), Pattern recognition techniques, Crane, London
  26. Wolberg, W. H. and Mangasarian, O. L. (1990), Multisurface method of pattern separation for medical diagnosis applied to breast cytology, Proceedings of the National Academy of Sciences, 87, 9193-9196