Browse > Article
http://dx.doi.org/10.9717/kmms.2016.19.3.642

Implementation of Muscular Sense into both Color and Sound Conversion System based on Wearable Device  

Bae, Myungjin (Dept. of Advanced Engineering, Kyungnam University)
Kim, Sungill (Dept. of Electronic Engineering, Kyungnam University)
Publication Information
Journal of Korea Multimedia Society / v.19, no.3, 2016 , pp. 642-649 More about this Journal
Abstract
This paper presents a method for conversion of muscular sense into both visual and auditory senses based on synesthetic perception. Muscular sense can be defined by rotation angles, direction changes and motion degrees of human body. Synesthetic interconversion can be made by learning, so that it can be possible to create intentional synesthetic phenomena. In this paper, the muscular sense was converted into both color and sound signals which comprise the great majority of synesthetic phenomena. The measurement of muscular sense was performed by using the AHRS(attitude heading reference system). Roll, yaw and pitch signals of the AHRS were converted into three basic elements of color as well as sound, respectively. The proposed method was finally applied to a wearable device, Samsung gear S, successfully.
Keywords
Wearable Device; Muscular Sense; Synesthetic Conversion; Color Model; MIDI;
Citations & Related Records
Times Cited By KSCI : 3  (Citation Analysis)
연도 인용수 순위
1 G.T. Kim, K.S. Lee, and K.J Lee, “IoT Technology Trends Based on Wearable Devices,” Journal of Korea Contents Association, Vol. 13, No. 1, pp. 25-30, 2015.
2 M.G. Kim, Development of Wearable Device Input Interface by Electromyographic Signal Measurement, Master's Thesis of Inje University, 2015.
3 S. Patel, H. Park, P. Bonato, L. Chan, and M. Rodgers, “A Review of Wearable Sensors and Systems with Application in Rehabilitation,” Journal of Neuroengineering and Rehabilitation, Vol. 9, No. 12, pp. 1-17, 2012.   DOI
4 T.H. Yoon, S.W. Yoon, J.H. Ko and H.K Kim, “Research and Design of Smart Phone Sensor-based Context-aware System,” Journal of Korea Multimedia Society, Vol. 18, No. 3, pp. 408-418, 2015.   DOI
5 H.Y. Yoo, A Study on Color Recognition by Color Synesthesia, Master's Thesis of Ewha University, 2010.
6 K.Y. Jun, A Synesthetic Expression of Sound and Colors, Master's Thesis of Ewha University, 2004.
7 Y.C. Kim. A Study on the Factors That Create Synesthesia-Focusing on Relationship between the Senses of Touch, Sound, and Color, Doctoral Thesis of Hong-Ik University, 2005.
8 R. Rouw and H.S. Scholte, "Neural Basis of Individual Differences in Synesthetic Experiences," Journal of Neuroscience, Vol. 30, No. 18, pp. 6205-6213, 2010.   DOI
9 M.J. Bae and S.I. Kim, “The System of Converting Muscular Sense Into both Color and Sound Based on the Synesthetic Perception,” Journal of Korean Institute of Intelligent Systems, Vol. 24, No. 5, pp. 462-469, 2014.   DOI
10 M. Edward and S. Ramachandran. "Neurocognitive Mechanisms of Synesthesia," Journal of Neuron, Vol. 48, No. 3, pp. 509-520, 2005.   DOI
11 M.E. Hubbard, A.C. Arman, V.S. Ramachandran and G.M. Boynton, "Individual Differences among Grapheme-color Synesthetes: Brain-Behavior Correlations," Journal of Neuron, Vol. 45, No. 6, pp. 975-985, 2005.   DOI
12 J.Y. Kim, “A Study on Applied Synesthesia Visualization of Brand Design,” Journal of Academia-industrial Technology, Vol. 2, No. 97, pp. 874-877, 2011.
13 H.Y. Jeon, A Study on Musical Appreciative Guidance Utilizing Synesthesia In The Middle School-Mainly Focused on Scheherazade by Rimsky Korsakov, Master's Thesis of Yonsei University, 2014.
14 Y.J. Seo, Tizen to Develop a Wearable App, Hanbitmedia Publishers, Seoul, 2015.
15 S.K. Suslick, "Synesthesia in Science and Technology: More Than Making the Unseen Visible," Journal of Current Opinion in Chemical Biology, Vol. 16. No. 5, pp. 557-563, 2012.   DOI
16 B. Daniel, R. Nicolas, J. David, and C. Stephanie, Adults Can Be Trained To Acquire Synesthetic Experiences, Scientific Reports 4, 2014.
17 N. Witthoft and J. Winawer, "Learning, Memory, and Synesthesia," Journal of Psychological Science, Vol. 24, No. 3, pp. 258-265, 2013.   DOI
18 J.H. Kim, The Upper Limb Position Sense of the Athletes in Different Sport Events, Master's Thesis of Korea National University of Education, 2004.
19 O.C. Heo, Attitude Estimation of the Moving Bodies Using the MEMS Sensor, Master's Thesis of Sungkyunkwan University, 2010.
20 Praat, http://www.praat.org (accessed Feb., 24, 2016)