Browse > Article
http://dx.doi.org/10.7776/ASK.2013.32.3.214

Review of Standard Sound Quality Assessment Methods for the Transmitted and Processed Sounds  

Oh, Wongeun (Department of Multimedia Engineering, Sunchon National University)
Abstract
Assessing the quality of audio signals is an important consideration in making high quality sounds and various methods have been developed. This paper provides a general framework of sound quality and a technical overview of the international standard methods which are described in ITU-T, ITU-R, IEC and ANSI Recommendations in the speech intelligibility, speech quality, and audio quality areas. In addition, some recent findings and future works are included.
Keywords
International Standards; Sound quality assessment; Speech intelligibility; Speech quality; Audio quality;
Citations & Related Records
연도 인용수 순위
  • Reference
1 ITU-R BS.1534-1, Method for the Subjective Assessment of intermediate Quality Level of Coding Systems, 2001.
2 ITU-R BS.1387-1, Method for Objective Measurements of Perceived Audio Quality,1998.
3 ANSI S3.2-1989 (R1999), Method for Measuring the Intelligibility of Speech over Communication Systems, 1999.
4 ANSI S3.5-1997 (R2012), American National Standard Methods for Calculation of the Speech Intelligibility Index, 2012.
5 IEC 60268-16, Sound System Equipment-Part 16: Objective Rating of Speech Intelligibility by Speech Transmission Index, 2011.
6 M. Bodden, "Perceptual sound quality evaluation," in Proc. InterNoise2000, 1-6 (2000).
7 H. Fastl, Psychoacoustics and sound quality, in communication acoustics, edited by J. Blauert (Springer-Verlag, Berlin, 2005).
8 ISO 3382-2, Acoustics-Measurement of Room Acoustic parameters-Part 2: Reverberation Time in Ordinary Rooms, 2008.
9 Y. Huang and J. Benesty(Eds.), Audio Signal Processing for Next-Generation Multimedia Communication Systems (Kluwer Academic Publishers, Norwell, 2004).
10 P. C. Loizou, Speech Enhancement (CRC Press, Boca Raton, 2007).
11 P. Kabal, An examination and interpretation of ITU-R BS.1387: perceptual evaluation of audio quality, (McGill Univ., Rep., 2003).
12 SFPE editor, "Speech intelligibility," Fire Protection Engineering, 16-18 (2002).
13 Speech Intelligibility Papers, http://www.meyersound.com/ support/papers/speech/index.htm, 2013
14 N. A. Geoffroy, Measuring Speech Intelligibility in Voice Alarm Communication Systems (MS thesis, Worcester Polytechnic Institute, 2005).
15 S.-W. Byun, "Frequencies of Korean phonemes and reliability of Korean phonetically balanced word lists,"(in Korean), Kr. J. Otol. 44, 485-489 (2001).
16 S.-W. Byun, S. M. Chung, H. S. Kim, and Y. M. Go, "A Survey of phonetically balanced words lists used in training hospitals in Korea," (in Korean), Kr. J. Otol. 48, 1086-1090 (2005).   과학기술학회마을
17 T. Y. Hahm, "Complementary study on construction of Korean word lists for speech audiometry," (in Korean), Inje Med. J. 7, 1-19 (1986).
18 C. S. Yoon, S. W. Kim, and Y. K. Oh, "A study on the standardization of articulation testing method and its evaluation suitable for Korean language (I)," (in Korean), J.Arch. Instit. Kr. 4, 117-125 (1988).
19 C. S. Yoon, S. W. Kim, and Y. K. Oh, "A study on the standardization of articulation testing method and its evaluation suitable for Korean language(I)," (in Korean), J.Arch. Instit. Kr. 5, 95-108 (1989).
20 B. D. Jun, N. Choi, H.-W. Ko, and K. Sung, "Intelligent diagnostics for sound reproduction system by the use of PEAQ," Adv. in Neural Networks-ISNN, 382-389 (2006).
21 B. Feiten and I. Wolf, "Audio adaptation according to usage environment and perceptual quality metrics," IEEE Trans. on Multimedia 7, 446-453 (2005).   DOI   ScienceOn
22 E. S. Myakotnykh and S. U. Peter, "Towards a computational quality model for IP-based audio," in Proc. QoMEX, 110-115 (2009).
23 B. C. J. Moore and C.-T. Tan, "Perceived naturalness of spectrally distorted speech and music," J. Acoust. Soc. Am.114, 408-419 (2003).   DOI   ScienceOn
24 F. Rumsey, "Spatial quality evaluation for reproduced sound: terminology, meaning, and a scene-based paradigm," J. Acoust. Soc. Am. 50, 651-666 (2002).
25 B. C. J. Moore, C.-T. Tan, N. Zacharov, and V.-V. Mattila, "Measuring and predicting the perceived quality of music and speech subjected to combined linear and nonlinear distortion," J. Audio Eng. Soc. 52, 1228-1244 (2004).
26 C.-T. Tan, B. C. J. Moore, N. Zacharov, and V.-V. Mattila, "Predicting the perceived quality of nonlinearly distorted music and speech signals," J. Audio Eng. Soc. 52, 699- 711(2004).
27 J. A. N. Berg and F. Rumsey, "Systematic evaluation of perceived spatial quality," in AES 24th Intern. Conf. on Multichannel Audio, paper no. 43 (2003).
28 F. Rumsey, and S. Bech, "On the relative importance of spatial and timbral fidelities in judgements of degraded multichannel audio quality," J. Acoust. Soc. Am. 118, 968-976 (2005).   DOI   ScienceOn
29 S. Zielinski, "On some biases encountered in modern listening tests," in Spatial Audio & Sensory Eval. Tech. (2006).
30 J. Blauert and U. Jekosch, "Concepts behind sound quality: some basic considerations," in InterNoise2003 (2003).
31 ISO 3382-1, Acoustics-Measurement of Room Acoustic parameters-Part 1: Performance Spaces, 2009.
32 W. Li and R. F. Kubichek, "Output-based objective speech quality measurement using continuous hidden Markov models," in Proc. 7th Intern. Sym. Sig. Proc. and Its Appl., 389-392 (2003).
33 P. Gray, M. Hollier, and R. Massara, "Non-intrusive Speech Quality Assessment Using Vocal Track Models," Inst. Elect. Eng. Proc. Vis. Img. Sig. Proc. 147, 493-501 (2000).
34 F. Rumsey, "Subjective assessment of the spatial attributes of reproduced sound," in Proc. AES 15th Intern. Conf., 122-135 (1998).
35 D. Kim, "ANIQUE: an auditory model for single-ended speech quality estimation," IEEE Trans. on Speech and Audio Proc.13, 821-831 (2005).   DOI   ScienceOn
36 T.-Y. Yen, J.-H. Chen, and T.-S. Chi, "Perception-based objective speech quality assessment," in Proc. ICASSP, 4521-4524 (2009).
37 ITU-T, P.Sup23 : ITU-T coded-speech database, 2004.
38 J. Berg, "Evaluation of perceived spatial audio quality," in Proc. 9th World MultiConf. on Syst. Cyber. and Inform., 10-14 (2005).
39 F. Rumsey, "Spatial audio and sensory evaluation techniques context, history and aims," in Proc. Spatial Audio & Sensory Eval. Tech., 1-7 (2006).
40 S. H. Park, S. W. Ryu, J. Y. Park, and J. Shin, "Analysis and evaluation of PEAQ: Objective method for perceived audio quality measurement," (in Korean), in Proc. ITFE, 234-239 (2003).
41 M. Salovarda, I. Bolkovac, and H. Domitrovic, "Estimating perceptual audio system quality using PEAQ algorithm," in 18th Intern. Conf. on Appl. Electromag. and Comm., 1-4 (2005).
42 G. Markovic, Analysis of Methods for Objective Evaluation of Quality of Audio Signals and Application in Implementation of An Ecoder on A Class of Digital Signal Processors (Ph.D Thesis, University of Novi Sad, 2006).
43 S. Lee, N. Choi, and K. Sung, "A study on the subjective quality assessment of sound," Inform. Comm. Mag. 22, 1386-1396 (2005).
44 C. D. Creusere, K. D. Kallakuri, and R. Vanam, "An objective metric of human subjective audio quality optimized for a wide range of audio fidelities," IEEE Trans. on Audio Speech and Lang. Proc.16, 129-136 (2008).   DOI   ScienceOn
45 I. Choi, B. G. Shinn-Cunningham, S. B. Chon, K.-M. Sung, "Objective measurement of perceived auditory quality in multi-channel audio compression coding systems," J. Audio Eng. Soc. 56, 3-17 (2008).
46 J. G. A. Barbedo, A. Lopes, "A New cognitive model for objective assessment of audio quality," J. Audio Eng. Soc.53, 22-31 (2005).
47 L. Abanto, G. Kemper, and J. Telles, "A novel fuzzy logic-based metric for audio quality assessment: objective audio quality assessment," in Proc. Telecom. (CONATEL), 17-20 (2011).
48 S. Greorge, S. Zielinski, F. Rumsey, "Initial developments of an objective method for the prediction of basic audio quality for surround audio recordings," AES 120th Convention, paper no. 6686 (2006).
49 F. Rumsey, S. Zielinski, P. Jackson, M. Dewhirst, R. Conetta, S. George, S. Bech, D. Meares, "QESTRAL(Part 1): quality evaluation of spatial transmission and reproduction using an artificial listener," AES 125th Convention, paper no. 7595 (2008).
50 R. Conetta, Towards the automatic assessment of spatial quality in the reproduced sound environment (Ph.D Thesis, University of Surrey, 2011).
51 A. J. Manders, D. M. Simpson, and S. L. Bell, "Objective prediction of the sound quality of music processed by an adaptive feedback canceller," IEEE Trans. on Audio, Speech, and Lang. Proc. 20, 1734-1745 (2012).   DOI   ScienceOn
52 S. Kandadai, J. Hardin, and C. D. Creusere, "Audio quality assessment using the mean structural similarity measure," in Proc. ICASSP, 221-224 (2008).
53 Y. Yue, X. Xiang, and W. Yaodu, "A novel objective method for evaluating the quality of streaming audio," in Proc. IC-BNMT, 555-559 (2009).
54 Y. Huh and K. Oh, "Report on ITU-R SG6 meeting," TTA J.135, 129-131 (2011).
55 R. Vanam, "Scalable perceptual metric for evaluating audio quality," in Proc. Rec. of the Thirty-Ninth Asilomar Conf., 319-323 (2005).
56 R. Huber and B. Kollmeier, "PEMO-Q - A new method for objective audio quality assessment using a model of auditory perception," IEEE Trans. on Audio, Speech and Lang. Proc. 14, 1902-1911 (2006).   DOI   ScienceOn
57 J. C. Hardin and C. D. Creusere, "Objective Analysis of Temporally Varing Audio Quality Metrics," in Proc. 42nd Asilomar Conf., 1245-1249 (2008).
58 C. D. Creusere and J. C. Hardin, "Assessing the quality of audio containing temporally varying distortions," IEEE Trans. on Audio, Speech and Lang. Proc. 19,711-720 (2011).   DOI   ScienceOn
59 A. Spanias, T. Painter, and V. Atti, Audio Signal Processing and Coding (Wiley, Hoboken, 2007).
60 R. Vanam and C. D. Creusere, "Evaluating low bitrate scalable audio quality using advanced version of PEAQ and energy equalization approach," in Proc. ICASSP, 189-192 (2005).
61 ITU-T P.800, Methods for Subjective Determination of Transmission Quality, 1996.
62 ITU-T P.800.1, Mean Opinion Score(MOS) Terminology, 2006.
63 ITU-T P.862, Perceptual Evaluation of Speech Quality (PESQ): An Objective Method for End-to-End Speech Quality Assessment of Narrow-band Telephone Networks and Speech Codecs, 2001.
64 ITU-T P.862.1, Mapping Function for Transforming P.862 raw Result Scores to MOS-LQO, 2003.
65 ITU-T P.862.2, Wideband extension to Recommendation P.862 for the assessment of wideband telephone networks and speech codecs, 2007.
66 ITU-T P.862.3, Application Guide for Objective Quality Measurement Based on Recommendations p. 862, p. 862.1 and p. 862.2, 2007.
67 ITU-T P. 863, Perceptual Objective Listening Quality Assessment, 2011.
68 G. Ballou, Handbook for Sound Engineers, 4th ed., (Elsevier, Oxford, 2008).
69 S. H. Kang, Spatial Acoustics (Sound Media, Goyang, 2012).
70 L. Beranek, Concert Halls and Opera Houses, 2nd ed., (Springer-Verlag, New York, 2010).
71 F. A. Everest and K. C. Pohlmann, Master Handbook of Acoustics, 5th ed. (McGraw-Hill, New York, 2009).
72 Y. Hu, and P. C. Loizou, "Evaluation of objective quality measures for speech enhancement," IEEE Trans. on Audio, Speech, and Lang. Proc. 16, 229-238 (2008).
73 S. Willsallen, and D. Cabrera, "Assessment of music audio quality in a sports stadium," AES 117th Convention, paper no. 6273 (2004).
74 H. Fastl and E. Zwicker, Psychoacoustics-Facts and Models, 3rd ed. (Springer, Berlin, 2007).
75 H. J. Steeneken and T. Houtgast, "A physical method for measuring speech-transmission quality," J. Acoust. Soc. Am. 67, 318-326 (1980).   DOI   ScienceOn
76 T. Painter and A. Spanias, "Perceptual coding of digital audio," Proc. IEEE 88, 451-515 (2000).   DOI   ScienceOn
77 J. Kunio, "Using sound quality to improve your product," in Intern. Appl. Tech. Conf. & Ex., 1-14 (2006).
78 W. Hoeg, L. Christensen, and R. Walker, "Subjective assessment of audio quality-the means and methods with in the EBU," EBU Tech. Rev., 40-50 (1997).
79 S. Bech and N. Zacharov, Perceptual Audio Evaluation- Theory, Method and Application (Wiley, Atrium, 2006).
80 A. Rix, J. Beerends, D.-S. Kim, P. Kroon, and O. Ghitza, "Objective assessment of speech and audio quality technology and applications," IEEE Trans. on Audio Speech and Lang. Proc. 14, 1890-1901 (2006).
81 S. Zielinski, F. Rumsey, and S. Bech, "On some biases encountered in modern audio quality listening tests-A review," J. Audio Eng. Soc. 56, 427-451 (2008).
82 D. Campbell, E. Jones, and M. Glavin, "Audio quality assessment techniques-a review, and recent developments," Sig. Proc. 89, 1489-1500 (2009).
83 A. A. De Lima, F. P. Freeland, R. A. De Jesus, B. C. Bispo, L. W. P. Biscainho, S. L. Netto, a. a. de Lima, R. a. de Jesus, a. Said, a. Kalker, R. Schafer, B. Lee, and M. Jam, "On the quality assessment of sound signals," 2008 IEEE Inter. Sym. Cir. and Sys. 3, 416-419 (2008).
84 W. Oh and S.-K. Lee, "Quality assessment of sound signals in multimedia and communication systems," Comm. in Comp. and Inform. Science 353, 57-64 (2012).   DOI
85 J. H. Ku, Rayleigh's Acoustical Research (Korea Studies Information, Paju, 2008).
86 KS I ISO 8253-3:2009, Acoustics-Audiometric test methods- Part 3: Speech Audiometry, (in Korean), 2009.
87 H. Hermansky, "Perceptual linear prediction(PLP) analysis of speech," J. Acoust. Soc. Am. 87, 1738-1752 (1990).   DOI
88 ITU-T p. 563, Single ended Method for Objective Speech Quality Assessment in Narrow-band Telephony Applications, 2004.
89 ITU-R BS.1116-1, Methods for the Subjective Assessment of Small Impairments in Audio Systems Including Multichannel Sound Systems,1994.
90 ITU-R BS.1283-1, A Guide to ITU-R Recommendations for Subjective Assessment of Sound Quality,1997.
91 ITU-R BS.1284-1, General Methods for the Subjective Assessment of Sound Quality,1997.
92 ITU-R BS.1285, Pre-selection methods for the subjective assessment of small impairments in audio systems,1997.