Journal of Music and Human Behavior (인간행동과 음악연구)

Korean Music Therapy Education Association (한국음악치료교육학회)
권호 표지
DOI QR코드

DOI QR Code

Acoustic Analysis and Melodization of Korean Intonation for Language Rehabilitation

언어재활을 위한 한국어의 음향적 분석과 선율화

  • Choi, Jin Hee (Department of Music Therapy, Graduate School, Sungshin Women's University) ;
  • Park Jeong Mi (Department of Musicology, Seoul National University)
  • 최진희 (성신여자대학교 일반대학원 음악치료학과) ;
  • 박정미 (서울대학교 음악대학 음악학과)
  • Received : 2024.03.05
  • Accepted : 2024.04.15
  • Published : 2024.04.30
Download PDF
⟨ Previous Next ⟩

Abstract

This study aims to acoustically analyze Korean language characteristics and convert these findings into musical elements, providing foundational data for evidence-based music-language rehabilitation. We collected voice data from thirty men and thirty women aged 19-25, each providing six-syllable prosodic units composed of two accentual phrases, including both declarative and interrogative sentences. Analyzing this data with Praat, we extracted syllabic acoustic properties and conducted statistical analyses based on acoustic properties, sentence type, gender, and particle presence. Significant differences were found in syllable frequency and duration based on accentual phrases and prosodic units (p < .001), with interrogative showing higher frequencies and declaratives longer durations (p < .001). Female frequencies were significantly higher than males' (p < .001), with longer durations observed (p < .001). Particle syllables also showed significantly stronger intensities (p < .001). Finally, we presented melodies converted from these acoustic properties into musical scores based on pitch, duration, and accent. The insights from this analysis of six-syllable Korean sentences will guide further research on developing a system for melodizing large-scale Korean speech data, expected to be crucial in music-based language rehabilitation.

본 연구의 목적은 한국어의 특성을 음향학적으로 분석하고, 해당 분석 결과를 음악적 요소로 변환하여 생성된 선율의 예시를 제시함으로써, 근거 기반 음악-언어재활 프로그램 개발을 위한 기초 자료를 마련하는 데 있다. 이를 위해 19-25세의 남성과 여성 각각 30명으로부터 3음절의 두 강세구로 이루어진 6음절 한국어의 음성자료를 수집하였으며, 평서문과 의문문을 포함하였다. 이를 Praat 프로그램으로 분석하여 음절별로 음향적 속성을 추출하고, 각 음향적 속성, 문장 유형, 성별, 조사의 유무에 대한 통계분석을 실시하였다. 분석 결과 강세구, 억양구에 따라 각 음절의 주파수와 지속시간에 유의미한 차이가 발견되었고(p < .001), 의문문이 평서문보다 주파수가 유의미하게 높았으며(p < .001), 평서문이 유의미하게 긴 경향을 보였다(p < .001). 여성 집단이 남성 집단보다 주파수가 유의미하게 높았으며(p < .001), 지속시간도 유의미하게 긴 경향을 보였다(p < .001). 또한 조사인 음절의 강도가 유의미하게 세게 나타났다(p < .001). 마지막으로 본 연구에서는 문장 유형, 성별, 조사의 유무에 따라 나타난 음향적 속성을 음악적 요소인 음고, 음가, 악센트로 변환한 선율들을 악보로 제시하였다. 본 연구에서는 6음절 한국어문장을 대상으로 한 분석을 제시하였으며, 이 연구를 통해 얻은 통찰을 바탕으로 후속 연구에서는 대규모 한국어 음성 데이터의 선율화 시스템 개발 연구를 진행하려 하며, 이는 음악을 활용한 언어재활 분야의 기초 자료로서 중요한 역할을 할 수 있을 것으로 기대한다.

Keywords

Acknowledgement

본 논문은 정부(과학기술정보통신부)의 재원으로 한국연구재단의 지원을 받아 수행된 연구임 (No. RS-2023-00219429).

References

  1. Belin, P., Zibovicius, M., Remy, P., Francois, C., Guillaume, S., Chain, F., ... Samson, Y. (1996). Recovery from nonfluent aphasia after melodic intonation therapy: A PET study. Neurology, 47(6), 1504-1511. https://doi.org/10.1212/WNL.47.6.1504
  2. Bolinger, D. (1989). Intonation and its uses: Melody in grammar and discourse. Stanford, CA: Stanford University Press.
  3. Chow, I., & Brown, S. (2018). A musical approach to speech melody. Frontiers in Psychology, 9, 247.
  4. Cortese, M. D., Riganello, F., Arcuri, F., Pignataro, L. M., & Buglione, I. (2015). Rehabilitation of aphasia application of melodic-rhythmic therapy to Italian language. Frontiers in Human Neuroscience, 9, 520.
  5. Hart, J. T., Collier, R., & Cohen, A. (1990). A perceptual study of intonation: An experimental-phonetic approach to speech melody. Cambridge, UK: Cambridge University Press.
  6. Hong, H., & Kim, S. J. (2016). Vocal range of older adults in comparison with young adults depending on the presence of pitch cues. The Journal of the Korea Contents Association, 16(4), 377-386. https://doi.org/10.5392/JKCA.2016.16.04.377
  7. Hong, S. Y. (2010). A phonetic study on the prosodic realization of Korean English utterances. The Journal of Human Studies, 26, 243-276.
  8. Hough, M. S. (2010). Melodic intonation therapy and aphasia: Another variation on a theme. Aphasiology, 24(6-8), 775-786. https://doi.org/10.1080/02687030903501941
  9. Hwang, S. Y. (2014). A study on the awareness of advanced Korean learners on speakers' attitudes apparent through nuclear tones. Journal of Speech Communication, 25, 251-272. https://doi.org/10.18625/jsc.2014..25.251
  10. Jang, H. (2015). On the education contents of intonation for Korean language education. Korean Linguistics, 67, 193-215.
  11. Jun, S. A. (1993). The phonetics and phonology of Korean prosody (Unpublished doctoral dissertation). Ohio State University, Columbus.
  12. Jung, J., Lee, H. J., & Park, J. C. (2010). Intonation generation for Korean speech synthesis with automated sentence type classification. Proceedings of HCI Korea, 2010(1), 159-161.
  13. Jung, M., & Choi, E. (2013). A study on aspects of functions and intonations in uttering '-잖아' of Korean native speakers and Korean learners. Korean Linguistics, 60, 143-165.
  14. Jung, M., & Shin, J. (2000). A study on Korean time-Unit. Korean Linguistics, 12, 261-278.
  15. Kershenbaum, A., Nicholas, M. L., Hunsaker, E., & Zipse, L. (2019). Speak along without the song: What promotes fluency in people with aphasia. Aphasiology, 33(4), 405-428.
  16. Kim, C., Park, D., Kim, B., & Kwon, C. H. (1996). The rule of Korean pitch variation for a natural synthetic female voice. The Journal of the Acoustical Society of Korea, 15(6), 26-32.
  17. Kwon, S. M. (2010). A study of interlanguage intonation manifested in non-final endings that behave like final endings. Journal of Korean Language Education, 21(4), 1-23. https://doi.org/10.18209/IAKLE.2010.21.4.1
  18. Kwon, S. M. (2011). A study on the features of interlanguage prosody of Chinese learners of Korean: Focused on intonation of APs and boundary tones. Bilingual Research, 45(0), 1-25.
  19. Ladd, D. R. (2008). Intonational Phonology (2nd ed.). Cambridge, UK: Cambridge University Press.
  20. Lee, H. Y. (2011). A study on teaching advanced learners discrimination of the intonation of '-을걸' and the effectiveness of the teaching. Journal of Korean Language Education, 22(3), 257-283. https://doi.org/10.18209/IAKLE.2011.22.3.257
  21. Lee, Y. M., & Kim, H. H. (2001). An utterance analysis of conversations and picture description tasks of Korean adults. Communication Sciences and Disorders, 6(1), 40-52.
  22. Leonardi, S., Cacciola, A., De Luca, R., Aragona, B., Andronaco, V., Milardi, D., ... Calabro, R. S. (2018). The role of music therapy in rehabilitation: Improving aphasia and beyond. International Journal of Neuroscience, 128(1), 90-99. https://doi.org/10.1080/00207454.2017.1353981
  23. Lindsay, P. H., & Norman, D. A. (1972). Human information processing: An introduction of psychology. NY: Academic press.
  24. Miller, S. B., & Toca, J. M. (1979). Adapted melodic intonation therapy: A case study of an experimental language program for an autistic child. The Journal of Clinical Psychiatry, 40(4), 201-203.
  25. Nooteboom, S. (1997). The prosody of speech: Melody and rhythm. The Handbook of Phonetic Sciences, 5, 640-673.
  26. Oh, J. H. (2014). A study of methods of standardization for Korean intonation curve. Korean Linguistics, 62, 395-420.
  27. Oh, J. H. (2018). Current situation and tasks of research for Korean intonation education. Journal of Korean Culture, 43, 69-102.
  28. Park, K. Y. (2009). The study of teaching Korean final endings' intonation: A focus on intonation difference of modal meanings. The Studies of Korean Language and Literature, (34), 373-397.
  29. Patel, A. D. (2010). Music, language, and the brain. New York: Oxford University Press.
  30. Seki, K., & Sugishita, M. (1983). Japanese-applied melodic intonation therapy for broca aphasia. No To Shinkei: Brain and Nerve, 35(10), 1031-1037.
  31. Shin, D., & Lee, G. (2019). Prosodic features of Korean disyllabic words produced by English speaking learners of Korean. Studies in Phonetics, Phonology, and Morphology, 25(1), 57-71. https://doi.org/10.17959/SPPM.2019.25.1.57
  32. Shin, J., & Cha, J. (2003) The system of Korean sounds: Foundations for the study of Korean phonology. Seoul: Hanguk Munhwasa.
  33. Stainsby, T., & Cross, I. (2009). The perception of pitch. In S. Hallam, I. Cross, & M. Thaut (Eds.), Oxford handbook of music psychology (pp. 47-58). Oxford: Oxford University Press.
  34. Steele, J. (1775). An essay towards establishing the melody and measure of speech to be expressed and perpetuated by certain symbols. London: Bowyer and Nichols.
  35. Tabei, K. I., Satoh, M., Nakano, C., Ito, A., Shimoji, Y., Kida, H., & Tomimoto, H. (2016). Improved neural processing efficiency in a chronic aphasia patient following melodic intonation therapy: A neuropsychological and functional MRI study. Frontiers in Neurology, 7, 148.
  36. Thaut, M. H. (2009). Rhythm, music, and the brain: Scientific foundations and clinical applications (Y. A. Cha, Trans). Seoul: Hakjisa. (Original work published 2005)
  37. Weber-Fox, C., & Hampton, A. (2008). Stuttering and natural speech processing of semantic and syntactic constraints on verbs. Journal of Speech, Language, Hearing Research, 51(5), 1058-1071.
  38. Whetstone, T., Kim, S., Reynolds, C., & Deeves, E. (2018). Case study: Differential effects of melody and rhythm in melodic intonation therapy. Clinical Archives of Communication Disorders, 3(2), 127-136.
  39. Yi, S. Y., Oh, J., & Chong, H. J. (2016). Tonal characteristics based on intonation pattern of the Korean emotion words. Journal of Music and Human Behavior, 13(2), 67-83. https://doi.org/10.21187/JMHB.2016.13.2.67
  40. Yu, M. R. (2014). A study on phonological features of lyrics for Korean popular song writing. Lingua Humanitatis, 16(3), 105-126. https://doi.org/10.16945/201416305
  41. Yune, Y. (2020). Perceptive evaluatin of Korean native speakers on the polysemic sentence final ending produced by Chinese Korean learners. Phonetics and Speech Sciences, 12(4), 27-36. https://doi.org/10.13064/KSSS.2020.12.4.027