• Journal of the Ergonomics Society of Korea
• /
• v.23 no.3
• /
• pp.89-99
• /
• 2004

A comprehensive psychophysical scaling system, multiple-modality magnitude estimation system (MMES) has been designed to measure subjective multidimensional human perception. Unlike paper-based magnitude estimation systems, the MMES has an additional auditory peripheral cue that varies with corresponding visual magnitude. As the simplest, purely psychological case, bimodal divided-attention conditions were simulated to establish the superiority of the MMES. Subjects were given brief presentations of pairs of simultaneous stimuli consisting of visual line-lengths and auditory white-noise levels. In the visual or auditory focused-attention conditions, only the line-lengths or the noise levels perceived should be reported respectively. On the other hand, in the divided-attention conditions, both the line-lengths and the noise levels should be reported. There were no significant differences among the different attention conditions. Human performance was better when the proportion of magnitude in stimulus pairs were identically presented. The additional auditory cues in the MMES improved the correlations between the magnitude of stimuli and MMES values in the divided-attention conditions.

• MALSORI
• /
• no.65
• /
• pp.153-168
• /
• 2008

We propose a new method to apply shape analysis techniques to a computational auditory scene analysis (CASA)-based speech separation system. The conventional CASA-based speech separation system extracts speech signals from a mixture of speech and noise signals. In the proposed method, we complement the missing speech signals by applying the shape analysis techniques such as labelling and distance function. In the speech separation experiment, the proposed method improves signal-to-noise ratio by 6.6 dB. When the proposed method is used as a front-end of speech recognizers, it improves recognition accuracy by 22% for the speech-shaped stationary noise condition and 7.2% for the two-talker noise condition at the target-to-masker ratio than or equal to -3 dB.

• Science of Emotion and Sensibility
• /
• v.26 no.4
• /
• pp.51-70
• /
• 2023

This study employed driving simulation to examine how takeover request (TOR) information modalities (visual, auditory, and visual + auditory) in Level-3 automated vehicles, and road types (straight and curved) influence the driver's control takeover time (TOT) and mental workload, assessed through subjective workload and heart rate variations. The findings reveal several key points. First, visual TOR resulted in the quickest TOT, while auditory TOR led to the longest. Second, TOT was considerably slower on curved roads compared to straight roads, with the greatest difference observed under the auditory TOR condition. Third, the auditory TOR condition generally induced lower subjective workload and heart rate variability than the visual or visual + auditory conditions. Finally, significant heart rate changes were predominantly observed in curved road conditions. These outcomes indicate that TOT and mental workload levels in drivers are influenced by both the TOR modality and road geometry. Notably, a faster TOT is associated with increased mental workload.

• Science of Emotion and Sensibility
• /
• v.26 no.1
• /
• pp.43-54
• /
• 2023

In this study, we assessed how participants represent various sensory stimuli experiences through behavioral ratings and physiological measurements. Utilizing intersubject correlation (ISC) analysis, we evaluated whether individuals' affective responses of dominance, arousal, and valence differed when stimuli of three modality conditions (auditory, visual, and haptic) were presented. ISC analyses were used to measure the similarities between one participant's responses and those of the others. To calculate the intersubject correlation, we divided the entire dataset into one subject and all other subject datasets and then correlated the two for all possible stimulus pair combinations. The results revealed that for dominance, ISCs of the visual modality condition were greater than the auditory modality condition, whereas, for arousal, the auditory condition was greater than the visual modality. Last, negative valence conditions had the greater consistency of the participants' reactions than positive conditions in each of the sensory modalities. When comparing modalities, greater ISCs were observed in haptic modality conditions than in visual and auditory modality conditions, regardless of the affective categories. We discussed three core affective representations of multiple modalities and proposed ISC analysis as a tool for examining differences in individuals' affective representations.

• The Transactions of The Korean Institute of Electrical Engineers
• /
• v.60 no.6
• /
• pp.1196-1203
• /
• 2011

The reduced frequency selectivity is a typical phenomenon of sensorineural hearing loss. In this paper, we compared two modeling methods for reduced frequency selectivity of hearing impaired people. The two models of reduced frequency selectivity were made using LPC(linear prediction coding) algorithm and bandwidth control algorithm based on ERB(equivalent rectangular bandwidth) of auditory filter, respectively. To compare the effectiveness of two models, we compared the result of PESQ (perceptual evaluation of speech quality) and LLR(log likelihood ratio) using 36 Korean words of two syllables. To verify the effect on noise condition, we mixed white and babble noise with 0dB and -3dB SNR to speech words. As the result, it is confirmed that the PESQ score of bandwidth control algorithm is higher than the score of LPC algorithm, on the other hands, and the LLR score of LPC algorithm is lower than the score of bandwidth control algorithm. It means that both non-linearity and widen auditory filter characteristics caused by reduced frequency selectivity could be more reflected in bandwidth control algorithm than in LPC algorithm.

• Korean Journal of Child Studies
• /
• v.38 no.3
• /
• pp.75-90
• /
• 2017

Objective: This study aimed to identify children's cognitive processing and performance characteristics while multiple task performance. It confirmed whether their multiple task performance and psychological refractory period (PRP) varied by task condition (stimulus onset asynchrony [SOA] and task difficulty) and stimulus modality. Methods: Seventy 5-year-olds were recruited. Multi-task tools were developed using the E-prime software. The children were required to respond to two stimuli (visual or auditory) presented with microscopic time difference and their response times (RTs) were recorded. Results: As the SOA increased, the RTs in the first task increased, while the RTs in the second task and PRP decreased. The RTs of the first and second tasks, and the PRP for difficult tasks, were significantly longer than those for easy tasks were. Additionally, there was an interaction effect between the SOA and task difficulty. Although there was no main effect of stimulus modality, task difficulty moderated the modality effect. In the high difficulty condition, the RTs of the first and second tasks and PRP for the visual-visual task were significantly longer than those for auditory-auditory task were. Conclusion: These results inform theoretical discussions on children's multi-task mechanism, and the loss of multiple task performance. Additionally, they provide practical implications and information on the composition of multi-tasks suitable for children in educational environments.

• Journal of Audiology & Otology
• /
• v.23 no.1
• /
• pp.39-48
• /
• 2019

Background and Objectives: The influence of visual stimulus on the auditory component in the perception of auditory-visual (AV) consonant-vowel syllables has been demonstrated in different languages. Inherent properties of unimodal stimuli are known to modulate AV integration. The present study investigated how the amount of McGurk effect (an outcome of AV integration) varies across three different consonant combinations in Kannada language. The importance of unimodal syllable identification on the amount of McGurk effect was also seen. Subjects and Methods: Twenty-eight individuals performed an AV identification task with ba/ga, pa/ka and ma/ṇa consonant combinations in AV congruent, AV incongruent (McGurk combination), audio alone and visual alone condition. Cluster analysis was performed using the identification scores for the incongruent stimuli, to classify the individuals into two groups; one with high and the other with low McGurk scores. The differences in the audio alone and visual alone scores between these groups were compared. Results: The results showed significantly higher McGurk scores for ma/ṇa compared to ba/ga and pa/ka combinations in both high and low McGurk score groups. No significant difference was noted between ba/ga and pa/ka combinations in either group. Identification of /ṇa/ presented in the visual alone condition correlated negatively with the higher McGurk scores. Conclusions: The results suggest that the final percept following the AV integration is not exclusively explained by the unimodal identification of the syllables. But there are other factors which may also contribute to making inferences about the final percept.

• Korean Journal of Audiology
• /
• v.23 no.1
• /
• pp.39-48
• /
• 2019

Background and Objectives: The influence of visual stimulus on the auditory component in the perception of auditory-visual (AV) consonant-vowel syllables has been demonstrated in different languages. Inherent properties of unimodal stimuli are known to modulate AV integration. The present study investigated how the amount of McGurk effect (an outcome of AV integration) varies across three different consonant combinations in Kannada language. The importance of unimodal syllable identification on the amount of McGurk effect was also seen. Subjects and Methods: Twenty-eight individuals performed an AV identification task with ba/ga, pa/ka and ma/ṇa consonant combinations in AV congruent, AV incongruent (McGurk combination), audio alone and visual alone condition. Cluster analysis was performed using the identification scores for the incongruent stimuli, to classify the individuals into two groups; one with high and the other with low McGurk scores. The differences in the audio alone and visual alone scores between these groups were compared. Results: The results showed significantly higher McGurk scores for ma/ṇa compared to ba/ga and pa/ka combinations in both high and low McGurk score groups. No significant difference was noted between ba/ga and pa/ka combinations in either group. Identification of /ṇa/ presented in the visual alone condition correlated negatively with the higher McGurk scores. Conclusions: The results suggest that the final percept following the AV integration is not exclusively explained by the unimodal identification of the syllables. But there are other factors which may also contribute to making inferences about the final percept.

• Korean Journal of Fisheries and Aquatic Sciences
• /
• v.35 no.6
• /
• pp.563-567
• /
• 2002

Developing base data on luring fish schools into netting position by the use of underwater audible sound on japanese parrot fish Oplegnathus fasciatus found in the coastal waters of Jeju Island, S. Korea. Auditory threshold was determined by the heartbeat condition technique using pure tones coupled with a delayed electric shock. The audible range of japanese parrot fish extended from 80 Hz to 500 Hz with a peak sensitivity at 200 Hz. The mean auditory thresholds at the frequencies of 80 Hz, 100 Hz, 200 Hz,300 Hz and 500 Hz were 104 dB, 95 dB, 91 dB, 99 dB and 113 dB, respectively. As the frequency became higher than 200 Hz, the auditory threshold increased almost linearly with increasing frequency. Critical ratios of fishes measured in the presence of masking noise in the spectrum level range of 69$\~$78 dB (0 dB re 1$\mu$Pa/$\sqrt{Hz}$) ranged from 21 dB to 40 dB at test frequencies. The noise spectrum level at the start of masking was about 70 dB within the test frequency range. The sound pressure level of 100$\~$200 Hz recognized by japanese parrot fish under the ambient noise is above 91 dB and the critical ratio for them is above 21 dB.

• Journal of the Institute of Convergence Signal Processing
• /
• v.13 no.1
• /
• pp.1-5
• /
• 2012

In this study, we constructed a virtual auditory display(VAD) that enables listener to move in a room freely. The VAD system was installed in a soundproof room($4.7m(W){\times}2.8m(D){\times}3.0m(H)$). The system consisted of a personal computer, a sound presentation device, and a three-dimensional ultrasound sensor system. This system acquires listener's location and position from a three-dimension ultrasonic sensor system covering the entire room. Localization was realized by convolving the sound source with head related transfer functions(HRTFs) on personal computer(PC). The calculated result is generated through a LADOMi(Localization Auditory Display with Opened ear-canal for Mixed Reality). The HRTFs used in the experiment were measured for each listener with loudspeakers constantly 1.5m away from the center of the listener' s head in an anechoic room. To evaluate the system performance, we experimented a search task of a sound source position in the condition that the listener is able to move all around the room freely. As a result, the positioning error of presented sound source was within 30cm in average for all listeners.