• The Journal of the Acoustical Society of Korea
• /
• v.28 no.3E
• /
• pp.109-117
• /
• 2009

Identity recognition in real environment with a reliable mode is a key issue in human computer interaction (HCI). In this paper, we present a robust person identification system considering score-based optimal reliability measure of audio-visual modalities. We propose an extension of the modified reliability function by introducing optimizing parameters for both of audio and visual modalities. For degradation of visual signals, we have applied JPEG compression to test images. In addition, for creating mismatch in between enrollment and test session, acoustic Babble noises and artificial illumination have been added to test audio and visual signals, respectively. Local PCA has been used on both modalities to reduce the dimension of feature vector. We have applied a swarm intelligence algorithm, i.e., particle swarm optimization for optimizing the modified convection function's optimizing parameters. The overall person identification experiments are performed using VidTimit DB. Experimental results show that our proposed optimal reliability measures have effectively enhanced the identification accuracy of 7.73% and 8.18% at different illumination direction to visual signal and consequent Babble noises to audio signal, respectively, in comparison with the best classifier system in the fusion system and maintained the modality reliability statistics in terms of its performance; it thus verified the consistency of the proposed extension.

• Journal of Korean Academy of Nursing
• /
• v.30 no.1
• /
• pp.98-109
• /
• 2000

The purpose of this study was to identify the effect on improvement of the Activity of Daily Living (ADL) and decrease the cognitive function and agitation behaviors by reminiscence with audio-visual stimulation for senile dementia. The quasi-experimental design was used in this study. Subjects were 26 with mild senile dementia who were cared for at a Day Care Center for Dementia in Seoul. The data were collected from March to July, 1999. Subjects were divided into three groups : Control Igroup with 10 subjects, reminiscence group(Control II group with 8 subjects), and reminiscence with audio-visual stimulation group(experimental group with 8 subjects). The Control I group got routine care as usual. Control II group participated in reminiscence sessions for one hour a day, five times a week , for a period of 4 weeks. The experimental group participated in reminiscence with audio-visual stimulation sessions for one hour a day, five times a week, for a period of 4 weeks. Instruments of this study were color photography with sound that was developed through an open questionnaire about events, objects, humans in action and animals that 100 Korean elderly over 60 would like to memorize. This was referred from the Sensory Stimuli Package by Namazi and Haynes(1994). The effects of treatment was evaluated through MMSE-K by Kwon & Park(1989). Also the Brief Cognitive Rating Scale(BCRS) by Reisberg et al(1983) for the cognitive function, through Agitation Inventory by Cohen- Mansfield and Colleague(1989) for behavioral response and through the Rapid Disability Rating Scale-2(RDRS-2) by Linn & Linn(1982) for the activity of daily living respectively. Data analysis was done using SPSS for $\chi$2- test, ANOVA, repeated measures ANOVA. The results were as follows : 1. Reminiscence with audio-visual stimulation did not improve cognitive function for senile dementia, but significantly improved verbal expression, the subscale of cognitive function. 2. Reminiscence with audio-visual stimulation reduced agitation behavior of experimental group significantly, but there was no significant difference between groups. 3. Reminiscence with audio-visual stimulation did not significantly effect the activity of daily living after treatment. In conclusion, it was shown that the reminiscence with audio-visual stimulation was an effective therapy to improve verbal expression and to reduce agitation behaviors of senile dementia. Further research with more indepth approach is needed, considering characteristic and level individualized for each senile dementia.

• Phonetics and Speech Sciences
• /
• v.3 no.2
• /
• pp.65-70
• /
• 2011

Recently, audio-visual speech recognition (AVSR) has been studied to cope with noise problems in speech recognition. In this paper we propose a novel method of deciding weighting factors for audio-visual information fusion. We adopt the particle swarm optimization (PSO) to weighting factor determination. The AVSR experiments show that PSO-based normalized confidence measures (NCM) improve the rejection performance of mis-recognized words by 33%.

• International Journal of Control, Automation, and Systems
• /
• v.5 no.1
• /
• pp.61-69
• /
• 2007

In this paper, we developed not only a reliable sound localization system including a VAD(Voice Activity Detection) component using three microphones but also a face tracking system using a vision camera. Moreover, we proposed a way to integrate three systems in the human-robot interaction to compensate errors in the localization of a speaker and to reject unnecessary speech or noise signals entering from undesired directions effectively. For the purpose of verifying our system's performances, we installed the proposed audio-visual system in a prototype robot, called IROBAA(Intelligent ROBot for Active Audition), and demonstrated how to integrate the audio-visual system.

• Archives of design research
• /
• v.14 no.3
• /
• pp.49-58
• /
• 2001

The communication designer in the multimedia age should have a consideration that the sound also can be an important expression element to transmit ones thought as well as the visual sense like color, shape and motion. As you already know, someones thought can be better understood when transmitted to others using the visual and auditory senses together than using the visual or auditory sense alone. The meeting of sight and hearing bases on the synesthesia. For example, low sound reminds a person of dark color and high sound usually reminds light color. And a percussion instrument reminds a person a circle and melody reminds a line. The visual and auditory senses In communication of the multimedia age should act as an independent expression element gotten out of the synchronizing that just serves simple sight and related sound at the same time. Interaction between sight and hearing arose a different emotion that cannot be happened just by one element alone. So, the programs of communication design education in this multimedia age should fulfill the requirement that it can develop ones expression ability through the understanding of interaction between sight and hearing. In this study, we suggest the education programs Classified into following categories; audio visual Gestaltung, audio visual moving graphics, audio visual design.

• Journal of Institute of Control, Robotics and Systems
• /
• v.13 no.8
• /
• pp.719-725
• /
• 2007

In this paper, we present an audio-visual speech recognition system for noise-robust human-computer interaction. Unlike usual speech recognition systems, our system utilizes the visual signal containing speakers' lip movements along with the acoustic signal to obtain robust speech recognition performance against environmental noise. The procedures of acoustic speech processing, visual speech processing, and audio-visual integration are described in detail. Experimental results demonstrate the constructed system significantly enhances the recognition performance in noisy circumstances compared to acoustic-only recognition by using the complementary nature of the two signals.

• Journal of Korean Society of Archives and Records Management
• /
• v.9 no.1
• /
• pp.33-50
• /
• 2009

ADD(Agency for Defense Development) has developed digital audio-visual archives management system to ensure easy access and long-term preservation for digital audio-visual archives. This paper covers total process of the system development and database management in the aspect of preservation and utilization by users' easy search through digitization of audio-visual archives. In detail, it contains system design for images and video data handling, standard workflow establishment, data quality, and metadata settings for database by converting an analog data into digital format. Also, this study emphasizes the importance of audio-visual archives management system through cost-effectiveness analysis.

• The Journal of the Korea Contents Association
• /
• v.19 no.9
• /
• pp.212-220
• /
• 2019

This study evaluates the extent to which the government's financial support policy, the Audio-visual investment fund, contributed to raising capital for Korean films. Audio-visual investment fund in the Korean film industry, which has been formed through the public sector support since 1999. The Audio-visual investment fund is a leading financial support policy for the Korean film industry, and began with the investment of the Small and Medium Business Administration and the Korean Film Council. It has become an important source of Korean film production costs and has spread to other cultural industry sectors, as a way of capital procurement for a start-up companies and cultural projects. This study reconstruct the data of the organizations such as the size of a new investment fund by public sector, the ratio of public capital contribution, the amount and number of investment in Korean films, investment multiplier compared to equity investment, and the internal return rate(IRR) of liquidation funds in the Korean film capital market from 1999 to 2017. The purpose of this project was to provide the basis for assessing the achievements of the Audio-visual investment fund policy in contributing to the growth of the film industry.

• Journal of the Korean Institute of Intelligent Systems
• /
• v.17 no.4
• /
• pp.519-525
• /
• 2007

With the high interest on robot technology and application, the research on artificial auditory systems for robot is very active. In this paper we discuss sound source localization and tracing based on audio-visual information. For video signals we use face detection based on skin color model. Also, binaural-based DOA is used as audio information. We integrate both informations using Kalman filter. The experimental results show that audio-visual person tracking Is useful, specially in the case that some informations are not observed.

• Journal of The Korean Association For Science Education
• /
• v.21 no.3
• /
• pp.516-528
• /
• 2001

In the educational study, the measure of EEG(brain waves) can be useful method to study the functioning state of brain during learning behaviour. This study investigated the changes of neuronal response according to four times repetition of audio-visual learning. EEG data at the prefrontal$(Fp_{1},Fp_{2})$ were obtained from twenty subjects at the 8th grade, and analysed quantitatively using FFT(fast Fourier transform) program. The results were as follows: 1) In the first audio-visual learning, the activities of $\beta_{2}(20-30Hz)$ and $\beta_{1}(14-19Hz)$ waves increased highly, but the activities of $\theta(4-7Hz)$ and $\alpha$ (8-13Hz) waves decreased compared with the base lines. 2). According to the repetitive audio-visual learning, the activities of $\beta_{2}$ and $\beta_{1}$ waves decreased gradually after the 1st repetitive learning. And, the activity of $\beta_{2}$ wave had the higher change than that of $\beta_{1}$ wave. 3). The activity of $\alpha$ wave decreased smoothly according to the repetitive audio-visual learning, and the activity of $\theta$ wave decreased radically after twice repetitive learning. 4). $\beta$ and $\theta$ waves together showed high activities in the 2nd audio-visual learning(once repetition), and the learning achievement increased highly after the 2nd learning. 5). The right prefrontal$(Fp_{2})$ showed higher activation than the left$(Fp_{1})$ in the first audio-visual learning. However, there were not significant differences between the right and the left prefrontal EEG activities in the repetitive audio-visual learning. Based on these findings, we can conclude that the habituation of neuronal response shows up in the repetitive audio-visual learning and brain hemisphericity can be changed by learning experiences. In addition, it is suggested once repetition of audio-visual learning be effective on the improvement of the learning achievement and on the activation of the brain function.