Journal of the Korea Society of Computer and Information (한국컴퓨터정보학회논문지)

Korean Society of Computer Information (한국컴퓨터정보학회)
권호 표지
DOI QR코드

DOI QR Code

A Computer-Assisted Pronunciation Training System for Correcting Pronunciation of Adjacent Phonemes

  • Lee, Jaesung (School of Computer Science and Engineering, Chung-Ang University)
  • Received : 2018.11.14
  • Accepted : 2019.01.19
  • Published : 2019.02.28
Download PDF
⟨ Previous Next ⟩

Abstract

Computer-Assisted Pronunciation Training system is considered to be a useful tool for pronunciation learning for students who received elementary level English pronunciation education, especially for students who have difficulty in correcting their pronunciation in front of others or who are not able to receive face-to-face training. The conventional Computer-Assisted Pronunciation Training system shows the word to the user, the user pronounces the word, and then the system provides phoneme or audio feedback according to the pronunciation of the user. In this paper, we propose a Computer-Assisted Pronunciation Training system that can practice on the varying pronunciation according to positions of adjacent phonemes. To achieve this, the proposed system is implemented by recommending a series of words by focusing on adjacent phonemes for simplicity and clarity. Experimental results showed that word recommendation considering adjacent phonemes leads to improvement of pronunciation accuracy.

Keywords

CPTSCQ_2019_v24n2_9_f0001.png 이미지

Fig. 1. Pronunciation accuracy of each phoneme averaged over 5 users at 1st round test

CPTSCQ_2019_v24n2_9_f0002.png 이미지

Fig. 2. Pronunciation accuracy of each phoneme averaged over 5 users at 2nd round test

CPTSCQ_2019_v24n2_9_f0003.png 이미지

Fig. 3. Improvement of pronunciation accuracy according to each phoneme averaged over 5 users

Table 1. Phonemes considered by our system

CPTSCQ_2019_v24n2_9_t0001.png 이미지

Table 2. Pronunciation accuracy of each user at each testing round

CPTSCQ_2019_v24n2_9_t0002.png 이미지

Table 3. Worst 10 phonemes with lowest average pronunciation accuracy at the 1st round test

CPTSCQ_2019_v24n2_9_t0003.png 이미지

Table 4. Top 10 phonemes in the viewpoint of improved pronunciation accuracy shown in Fig. 3

CPTSCQ_2019_v24n2_9_t0004.png 이미지

Table 5. Important Adjacent phonemes for User 1 and a list of words possibly recommended

CPTSCQ_2019_v24n2_9_t0005.png 이미지

Table 6. Important Adjacent phonemes for User 2 and a list of words possibly recommended

CPTSCQ_2019_v24n2_9_t0006.png 이미지

References

  1. M. Pennington and J. Richards, "Pronunciation revisited," TESOL quarterly, Vol. 20, No. 2, pp. 207-225, 1986 https://doi.org/10.2307/3586541
  2. J. Smith and B. Beckmann, "Improving pronunciation through Noticing-Reformulation Tasks, University College London, 2005
  3. S. Shaik, "Computer assisted English pronunciation training to undergraduate students," Journal of English Language and Literature, Vol. 4, No. 2, pp. 117-121, 2015
  4. H. Liao, Y. Guan, J. Tu, and J. Chen, "A prototype of an adaptive Chinese pronunciation training system," System, Vol. 45, No. 1, 2014
  5. R. Hincks, "Speech technologies for pronunciation feedback and evaluation," ReCALL, Vol. 15, No. 1, pp. 3-20, 2003 https://doi.org/10.1017/S0958344003000211
  6. C.-S. Park, "Understanding Artificial Intelligence Technology for Artificial Intelligence Humanities," Journal of AI Humanities, Vol. 1, No. 1, pp. 173-182, 2018
  7. G. Demenko, A. Wagner, N. Cylwik, "The use of speech technology in foreign language pronunciation training," Archives of Acoustics, Vol. 35, No. 3, pp. 309-329, 2010 https://doi.org/10.2478/v10168-010-0027-z
  8. G. Kartal, "Working with an imperfect medium: Speech recognition technology in reading practice," Journal of Educational Multimedia and Hypermedia, Vol. 15, No. 3, pp. 303-328, 2006
  9. X. Qian, H. Meng, and F. Soong, "Capturing L2 segmental mispronunciations with joint-sequence models in computer-aided pronunciation training," In proceedings of 7th International Symposium on Chinese Spoken Language Processing, pp. 84-88, Tainan, Taiwan, 2010
  10. K. Wong, W. Leung, W. Lo, and H. Meng, "Development of an articulatory visual-speech synthesizer to support language learning," In proceedings of 7th International Symposium on Chinese Spoken Language Processing, pp. 139-143, Tainan, Taiwan, 2010
  11. K. Wong, W. Lo, and H. Meng, "Allophonic variations in visual speech synthesis for corrective feedback in CAPT," In proceedings of IEEE International Conference on Acoustic, Speech and Signal Processing, pp. 5708-5711, Prague, Czech, 2011
  12. K. Goodman, "Reading: A psycholinguistic guessing game," Literacy Research and Instruction, Vol. 6, No. 4, pp. 126-135, 1967
  13. P. Gough, C. Juel, and P. Griffith, "Reading, spelling, and the orthographic cipher," Lawrence Erlbaum Associates, Inc, 1992
  14. H. Franco, H. Bratt, R. Rossier et al., "EduSpeak: A speech recognition and pronunciation scoring toolkit for computer-aided language learning applications," Language Testing, Vol. 27, No. 3, pp. 401-418, 2010 https://doi.org/10.1177/0265532210364408
  15. S. Witt and S. Young, "Phone-level pronunciation scoring and assessment for interactive language learning," Speech Communication, Vol. 30, No. 2, pp. 95-108, 2000 https://doi.org/10.1016/S0167-6393(99)00044-8
  16. X. Xi, D. Higgins, K. Zechner, and D. Williamson, "A comparison of two scoring methods for an automated speech scoring system," Language Testing, Vol. 29, No. 1, pp. 371-394, 2012 https://doi.org/10.1177/0265532211425673
  17. H. Liao, J. Chen, S. Chang, et al., "Decision tree based tone modeling with corrective feedbacks for automatic Mandarin tone assessment," In proceedings of 11th Annual Conference on the International Speech Communication Association, pp. 602-605, Chiba, Japan, 2010
  18. M. Harrison, W. Lau, H. Meng, and L. Wang, "Improving mispronunciation detection and diagnosis of learners' speech with context-sensitive phonological rules based on language transfer," In proceedings of 9th Annual Conference on International Speech Communication Association, pp. 2787-2790, Brisbane, Australia, 2008
  19. M. Harrison, W. Lo, X. Qian, and H. Meng, "Implementation of an extended recognition network for mispronunciation detection and diagnosis in computer-assisted pronunciation training, In proceedings of ISCA Workshop Speech and Language Technology in Education, pp. 45-48, Warrickshire, UK, 2009
  20. L. Wang, X. Feng, H. Meng, "Mispronunciation detection based on cross-language phonological comparisons," In proceedings of International Conference on Audio, Language and Image Processing, pp. 307-311, Shanghai, China
  21. A. Neri, C. Cucchiarini, H. Strik, and L. Boves, "The pedagogy-technology interface in computer assisted pronunciation training," Computer assisted language learning, Vol. 15, No. 5, pp. 441-467, 2002 https://doi.org/10.1076/call.15.5.441.13473
  22. F. Zhang F, C. Huang, F. Soong, M. Chu, and R. Wang, "Automatic mispronunciation detection for Mandarin," In proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 5077-5080, Las Vegas, USA, 2008
  23. J. Doremalen, C. Cucchiarini, H. Strik, "Automatic pronunciation error detection in non-native speech: The case of vowel errors in Dutch," The Journal of the Acoustical Society of America, Vol. 134, No. 2, pp. 1336-1347, 2013 https://doi.org/10.1121/1.4813304
  24. H. Strik, K. Truong, F. De Wet, C. Cucchiarini, "Comparing different approaches for automatic pronunciation error detection," Speech communication, Vol. 51, No. 10, pp. 845-852, 2009 https://doi.org/10.1016/j.specom.2009.05.007
  25. L. Wang, X. Feng, H. Meng, "Automatic generation and pruning of phonetic mispronunciations to support computer-aided pronunciation training, In proceeding of 9th Annual Conference on the International Speech Communication Association, pp. 1729-1732, Brisbane, Australia, 2008
  26. J. Zhao, H. Yuan, W. Leung, H. Meng, J. Liu, S. Xia, "Audiovisual synthesis of exaggerated speech for corrective feedback in computer-assisted pronunciation training," In proceedings of 2013 IEEE International Confernece on Acoustics, Speech and Signal Processing, pp. 8218-8222, Vancouver, Canada, 2013
  27. P. Badin, A. Ben Youssef, G. Bailly, F. Elisei, and T. Hueber, "Visual articulatory feedback for phonetic correction in second language learning. In proceedings of ISCA Workshop Speech and Language Technology in Education, pp. 1-10, Tokyo, Japan, 2010
  28. Y. Iribe, S. Manosavanh, K. Katsurada, R. Hayashi, C. Zhu, T. Nitta, "Generating Animated Pronunciation from Speech Through Articulatory Feature Extraction," In proceedings of 12th Annual Conference on International Speech Communication Association pp. 1617-1620, Florence, Italy, 2011
  29. W. Lo, A. Harrison, and H. Meng, "Statistical phone duration modeling to filter for intact utterances in a computer-assisted pronunciation training system, In proceedings of The 35th IEEE International Conference on Acoustics Speech and Signal Processing, pp. 5238-5241, Dallas, USA, 2010
  30. J. Lee, C.-H Lee, D.-W. Kim, B.-Y. Kang, "Smartphone-assisted pronunciation learning technique for ambient intelligence," IEEE Access, Vol. 5, No. 1, pp. 312-325, 2017 https://doi.org/10.1109/ACCESS.2016.2641474
  31. J. Schalkwyk, D. Beeferman, F. Beaufays et al. ""Your word is my command'': Google search by voice: a case study," In collection of Advances in speech recognition, pp. 61-90, 2010
  32. H. Koo, "A study of the effects of vowels on the pronunciation of English sibilants," Speech Science, Vol. 15, No. 1, pp. 31-38, 2008
  33. Y. Yun and N. Lee, "Research on the effect of pronunciation training of English unaspirated stops for Koreans," Language and Linguistics, Vol. 57, No. 1, pp. 141-158, 2012
  34. J. Kim, "Korean speakers' pronunciation and pronunciation training of English stops," Phonetics Speech Science, Vol. 2, No. 1, pp. 29-36, 2010
  35. H. Koo, "A study of production difficulties of English bilabial stops and labiodental fricateives by Korean learners of English," Phonetics Speech Science, Vol. 1, No. 1, pp. 11-15, 2009
  36. Y. Yun, "The learning effect of English vowels using the phonological information of Korean vowels," Journal of Modern British American Language Literature, Vol. 30, No. 1, pp. 75-91, 2012
  37. J. Kim and K. Yoon, "The formant frequency difference of English vowels as a function of stress and its application on vowel pronunciation training," Phonetics Speech Science, Vol. 5, No. 1, pp. 53-58, 2013
  38. K.-Y. La, "Improvement methods for teaching primary school English pronunciation in the EFL environment," Studies in English Education, Vol. 6, No. 2, pp. 5-31, 2001