Recognition of Superimposed Patterns with Selective Attention based on SVM

SVM기반의 선택적 주의집중을 이용한 중첩 패턴 인식

  • Bae, Kyu-Chan (Department of Electrical Engineering & Computer Science and Brain Science Research Center, Korea Advanced Institute of Science and Technology) ;
  • Park, Hyung-Min (Department of Electrical Engineering & Computer Science and Brain Science Research Center, Korea Advanced Institute of Science and Technology) ;
  • Oh, Sang-Hoon (Division of Information Communivation and Radio Engineering, Mokwon University) ;
  • Choi, Youg-Sun (Department of Biosystems and Brain Science Research Center, Korea Advanced Institute of Science and Technology) ;
  • Lee, Soo-Young (Department of Biosystems and Brain Science Research Center, Korea Advanced Institute of Science and Technology)
  • 배규찬 (한국과학기술원 전자전산학과 및 뇌과학연구센터) ;
  • 박형민 (한국과학기술원 전자전산학과 및 뇌과학연구센터) ;
  • 오상훈 (목원대학교 정보통신전파공학과) ;
  • 최용선 (한국과학기술원 바이오시스템학과 및 뇌과학연구센터) ;
  • 이수영 (한국과학기술원 바이오시스템학과 및 뇌과학연구센터)
  • Published : 2005.09.25

Abstract

We propose a recognition system for superimposed patterns based on selective attention model and SVM which produces better performance than artificial neural network. The proposed selective attention model includes attention layer prior to SVM which affects SVM's input parameters. It also behaves as selective filter. The philosophy behind selective attention model is to find the stopping criteria to stop training and also defines the confidence measure of the selective attention's outcome. Support vector represents the other surrounding sample vectors. The support vector closest to the initial input vector in consideration is chosen. Minimal euclidean distance between the modified input vector based on selective attention and the chosen support vector defines the stopping criteria. It is difficult to define the confidence measure of selective attention if we apply common selective attention model, A new way of doffing the confidence measure can be set under the constraint that each modified input pixel does not cross over the boundary of original input pixel, thus the range of applicable information get increased. This method uses the following information; the Euclidean distance between an input pattern and modified pattern, the output of SVM, the support vector output of hidden neuron that is the closest to the initial input pattern. For the recognition experiment, 45 different combinations of USPS digit data are used. Better recognition performance is seen when selective attention is applied along with SVM than SVM only. Also, the proposed selective attention shows better performance than common selective attention.

본 논문에서는 신경회로망보다 우수한 성능을 보이는 학습 이론인 SVM을 기반으로, 인간의 인지 과학에서 많은 연구가 이루어지고 있는 선택적 주의집중을 응용한 중첩 패턴 인식 시스템을 제안한다. 제안된 선택적 주의집중 모델은 SVM의 입력단에 주의집중층을 추가하여 SVM의 입력을 직접 변화시키는 학습을 하며 선택적 필터의 기능을 수행한다. 주의집중의 핵심은 학습을 멈추는 적절한 시점을 찾는 것과 그 시점에서 결과를 판단하는 주의집중 척도를 정의하는 것이다. 지지벡터는 주변에 존재하는 패턴들을 대표하는 표본이므로 입력 패턴이 초기상태일 때 주의집중을 하고자 하는 클래스의 가장 가까운 지지벡터를 기준으로 그 지지벡터와의 거리가 최소가 되었을 때 주의집중을 멈추는 것이 적절하다. 일반적인 주의집중을 적용하면 주의집중 척도를 정의하기가 난해해지기 때문에 변형된 입력이 원래 입력의 범위를 넘지 않는다는 제약조건을 추가하여 사용할 수 있는 정보의 폭을 넓히고 새로운 척도를 정의하였다. 이때 사용한 정보는 변형된 입력과 원래 입력의 유클리드 거리, SVM의 출력, 초기상태에 가장 가까웠던 히든뉴런의 출력값이다. 인식 실험을 위해 USPS 숫자 데이터를 사용하여 45개의 조합으로 중첩시켰으며, 주의집중을 적용시켰을 때 단일 SVM보다 인식 성능이 월등히 우수함을 확인하였고, 또한 제한된 주의집중을 사용하였을 때 일반적 주의집중을 이용하는 것 보다 성능이 더 뛰어났음을 확인하였다.

Keywords

References

  1. C. Bishop. Neural Networks for Pattern Recognition. Clarendon Press, London, 1995
  2. Vladimir N. Vapnik, 'The Nature of Statistical Learning Theory', Wiley, N.Y. ,pp. 131-170, 1995
  3. Christoper J. C. Burges, 'A Tutorial on Support Vector Machines for Pattern Recognition,' Data Mining and Knowledge Discovery, vo2, pp. 121-167, Kluwer Academic Publishers, Boston, 1998 https://doi.org/10.1023/A:1009715923555
  4. Corinna Cortes and Vladimir Vapnik, Support-Vector Networks, Machine Learning, vol 20, No 3, p273-297, 1995 https://doi.org/10.1007/BF00994018
  5. E. Cherry. Some experiments on the recognition of speech, with one and with two ears. Journal of the Acoustical Society of America, 25:975-979, 1953 https://doi.org/10.1121/1.1907229
  6. A. Treisman. Monitoring and storage of irrelevant messages in selective attention. Journal of Verbal Learning and Verbal Behavior, 3:449-459, 1964 https://doi.org/10.1016/S0022-5371(64)80015-3
  7. K. Fukushima. 'Neural network model for selective attention in visual pattern recognition and associative recall', Applied Optics, 26[23], pp. 4985-4992 Dec. 1987 https://doi.org/10.1364/AO.26.004985
  8. B. Scholkopf, K.-K. Sung, C. J. C. Burges, F. Girosi, P. Niyogi, T. Poggio, and V. Vapnik. Comparing Support Vector Machines with Gaussian Kernels to Radial Basis Function Classifiers. IEEE Transations on Signal Processing, 45:2758-2765, 1997 https://doi.org/10.1109/78.650102
  9. J. Weston and C. Watkins. Multi-class support vector machines. Technical Report CSD-TR-98-04, Department of Computer Science, Royal Holloway, University of London, Egham, TW20 0EX, UK, 1998
  10. M. Posner. Attention in cognitive neuroscience: An overview. In M. Gazzaniga et al., editor, The Cognitive Neurosciences, number V, pp. 615-624, MIT Press, 1966
  11. M. Posner and M. Raichle. Images of Mind, Scientific American Library, New York, 1994
  12. Ki-Young Park and Soo-Young Lee, Selective Attention for Robust Speech Recognition in Noisy Environments, International Joint Conference on Neural Networks, Washington, USA, July, 1999 https://doi.org/10.1109/IJCNN.1999.836021