Browse > Article
http://dx.doi.org/10.7730/JSCM.2022.34.3.31

Sasang Constitution Classification using Convolutional Neural Network on Facial Images  

Ahn, Ilkoo (KM Data Division, Korea Institute of Oriental Medicine)
Kim, Sang-Hyuk (KM Data Division, Korea Institute of Oriental Medicine)
Jeong, Kyoungsik (KM Data Division, Korea Institute of Oriental Medicine)
Kim, Hoseok (KM Data Division, Korea Institute of Oriental Medicine)
Lee, Siwoo (KM Data Division, Korea Institute of Oriental Medicine)
Publication Information
Journal of Sasang Constitutional Medicine / v.34, no.3, 2022 , pp. 31-40 More about this Journal
Abstract
Objectives Sasang constitutional medicine is a traditional Korean medicine that classifies humans into four constitutions in consideration of individual differences in physical, psychological, and physiological characteristics. In this paper, we proposed a method to classify Taeeum person (TE) and Non-Taeeum person (NTE), Soeum person (SE) and Non-Soeum person (NSE), and Soyang person (ST) and Non-Soyang person (NSY) using a convolutional neural network with only facial images. Methods Based on the convolutional neural network VGG16 architecture, transfer learning is carried out on the facial images of 3738 subjects to classify TE and NTE, SE and NSE, and SY and NSY. Data augmentation techniques are used to increase classification performance. Results The classification performance of TE and NTE, SE and NSE, and SY and NSY was 77.24%, 85.17%, and 80.18% by F1 score and 80.02%, 85.96%, and 72.76% by Precision-Recall AUC (Area Under the receiver operating characteristic Curve) respectively. Conclusions It was found that Soeum person had the most heterogeneous facial features as it had the best classification performance compared to the rest of the constitution, followed by Taeeum person and Soyang person. The experimental results showed that there is a possibility to classify constitutions only with facial images. The performance is expected to increase with additional data such as BMI or personality questionnaire.
Keywords
Sasang Constitution; Facial Images; Convolutional Neural Network; Deep Learning; Transfer Learning;
Citations & Related Records
Times Cited By KSCI : 8  (Citation Analysis)
연도 인용수 순위
1 Weiss K, Khoshgoftaar TM, Wang D. A survey of transfer learning. Journal of Big data. 2016 Dec;3(1):1-40. DOI: 10.1186/s40537-016-0043-6   DOI
2 Yoo HJ, Jun KK. Data Augmentation for Image based Parking Space Classification Deep Model. Journal of KIISE (JOK). 2022 Feb;49(2):126-36. DOI: 10.5626/JOK.2022.49.2.126   DOI
3 Hwang DS. Learning Behavior Analysis of Bayesian Algorithm Under Class Imbalance Problems. Journal of the Institute of Electronics Engineers of Korea CI. 2008;45(6):179-86.
4 Cho DU, Kim BH, Lee SH. Sasang Constitution classification system using face morphologic relation analysis. The KIPS Transactions: PartB. 2007; 14(3):153-62. (Korean). DOI: 10.3745/KIPSTB.2007.14-B.3.153   DOI
5 Am I a Taeumin? or a Soyangin?... Represent the Standard Facial Features in the Sasang Constitution. the Dong-A Daily News. 2012.02.13. Available from: URI: http://news.donga.com/3/all/20120113/43282152/1. (Korean).
6 LeCun Y, Bengio Y, Hinton G. Deep learning. nature. 2015 May;521(7553):436-44. DOI: 10.1038/nature14539   DOI
7 Gonzalez RC, Woods RE. Digital image processing, prentice hall. Upper Saddle River, NJ. 2008.
8 Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556. 2014 Sep 4. DOI: 10.48550/arXiv.1409.1556   DOI
9 Pham DD, Do JH, Ku B, Lee HJ, Kim H, Kim JY. Body mass index and facial cues in Sasang typology for young and elderly persons. Evidence-Based Complementary and Alternative Medicine. 2011 Jan 1;2011. DOI: 10.1155/2011/749209   DOI
10 Song HW, Park GC, Park J. Prediction of dairy cow mastitis with multi-sensor data using Multi-Layer Perceptron (MLP). In Proceedings of the Korea Information Processing Society Conference 2020 (pp. 788-791). Korea Information Processing Society. DOI: 10.3745/PKIPS.y2020m11a.788   DOI
11 Jang ES, Park KH, Baek YH, Lee SW, Kim SH, Joo JC. A Development of Sasang Constitution Diagnosis Program Based on Characteristics of Body Shape Face Physiological Symptom and Personality. Journal of Sasang Constitutional Medicine. 2012;24(1):21-31. (Korean). DOI: 10.7730/jscm.2012.24.1.021   DOI
12 Seok JH, Yoon JH, Lee JH, Hwang MW, Cho YJ, Kho BH et al. The upgrade of Sasangin classification system by the morphologic study of head and face: facial differences on sex and age. Journal of Sasang Constitutional Medicine. 2007;19(3): 30-50. (Korean).
13 Do JH, Jang E, Ku B, Jang JS, Kim H, Kim JY. Development of an integrated Sasang constitution diagnosis method using face, body shape, voice, and questionnaire information. BMC complementary and alternative medicine. 2012 Dec; 12(1):1-9. DOI: 10.1186/1472-6882-12-85   DOI
14 Lee J. Longevity and Life Preservation in Oriental Medicine (translated by SH Choi). Kyung Hee Univ. Press, Seoul; 1996. (Korean).
15 Lee HL, Cho JS. Sasang Constitution Classification System by Morphological Feature Extraction of Facial Images. Journal of The Korea Society of Computer and Information. 2015;20(8):15-21. (Korean). DOI: 10.9708/jksci.2015.20.8.015   DOI
16 Cho DU, Kim BH, Lee SH. Sasang Constitution Classification Using Shape Analysis of Face. In Proceedings of the Korea Information Processing Society Conference 2006 (pp. 423-426). Korea Information Processing Society. (Korean).
17 Lee SH, Kim BH, Ka MK, Cho DU, Kwak JH, Oh SY et al. A proposal of Sasang Constitution classification in middle-aged women using image and voice signals process. Journal of the Korea Academia-Industrial cooperation Society. 2008; 9(5):1210-7. (Korean). DOI: 10.5762/KAIS.2008.9.5.1210   DOI
18 Lee SY, Koh BH, Lee EJ, Lee JH, Hwang MW. Systematic review on researches of sasang constitution diagnosis using facial feature. Journal of Sasang Constitutional Medicine. 2012;24(4): 17-27. (Korean). DOI: 10.7730/JSCM.2012.24.4.17   DOI
19 Jung C, Yeo I, Jung H. Classification model of facial acne using deep learning. Journal of the Korea Institute of Information and Communication Engineering. 2019;23(4):381-7. (Korean). DOI:10.6109/jkiice.2019.23.4.381   DOI
20 Baek YH, Jin HJ, Kim HS, Jang ES, Lee SW. An Overview on the Construction of Korea Constitutional Multicenter Bank for Sasang Constitutional Medicine. Journal of Sasang Constitutional Medicine. 2012;24(2):47-53. (Korean). DOI:10.7730/JSCM.2012.24.2.047   DOI
21 King DE. Dlib-ml: A machine learning toolkit. The Journal of Machine Learning Research. 2009 Dec 1;10:1755-8.
22 D. S. Ma, J. Correll, and B. Wittenbrink, "The Chicago face database: A free stimulus set of faces and norming data.," Behav. Res. Methods, vol. 47, pp. 1122-1135, 2015.   DOI