• Journal of the Korean Society for information Management
• /
• v.41 no.1
• /
• pp.261-282
• /
• 2024

This study proposed and evaluated electroencephalography (EEG)-based and eye-tracking-based methods to determine relevance by utilizing users' implicit relevance feedback while navigating content in a digital library. For this, EEG/eye-tracking experiments were conducted on 32 participants using video, image, and text data. To assess the usefulness of the proposed methods, deep learning-based artificial intelligence (AI) techniques were used as a competitive benchmark. The evaluation results showed that EEG component-based methods (av_P600 and f_P3b components) demonstrated high classification accuracy in selecting relevant videos and images (faces/emotions). In contrast, AI-based methods, specifically object recognition and natural language processing, showed high classification accuracy for selecting images (objects) and texts (newspaper articles). Finally, guidelines for implementing a digital library interface based on EEG, eye-tracking, and artificial intelligence technologies have been proposed. Specifically, a system model based on implicit relevance feedback has been presented. Moreover, to enhance classification accuracy, methods suitable for each media type have been suggested, including EEG-based, eye-tracking-based, and AI-based approaches.

• Journal of Service Research and Studies
• /
• v.14 no.1
• /
• pp.13-26
• /
• 2024

Multi-modal generation is the process of generating results based on a variety of information, such as text, images, and audio. With the rapid development of AI technology, there is a growing number of multi-modal based systems that synthesize different types of data to produce results. In this paper, we present an AI system that uses speech and text recognition to describe a person and generate a montage image. While the existing montage generation technology is based on the appearance of Westerners, the montage generation system developed in this paper learns a model based on Korean facial features. Therefore, it is possible to create more accurate and effective Korean montage images based on multi-modal voice and text specific to Korean. Since the developed montage generation app can be utilized as a draft montage, it can dramatically reduce the manual labor of existing montage production personnel. For this purpose, we utilized persona-based virtual person montage data provided by the AI-Hub of the National Information Society Agency. AI-Hub is an AI integration platform aimed at providing a one-stop service by building artificial intelligence learning data necessary for the development of AI technology and services. The image generation system was implemented using VQGAN, a deep learning model used to generate high-resolution images, and the KoDALLE model, a Korean-based image generation model. It can be confirmed that the learned AI model creates a montage image of a face that is very similar to what was described using voice and text. To verify the practicality of the developed montage generation app, 10 testers used it and more than 70% responded that they were satisfied. The montage generator can be used in various fields, such as criminal detection, to describe and image facial features.

• Journal of Broadcast Engineering
• /
• v.25 no.6
• /
• pp.973-979
• /
• 2020

This study aims to objectively and quantitatively determine the social emotion of empathy by collecting pupillary response. 52 subjects (26 men and 26 women) voluntarily participated in the experiment. After the measurement of the reference of 30 seconds, the experiment was divided into the task of imitation and spontaneously self-expression. The two subjects were interacted through facial expressions, and the pupil images were recorded. The pupil data was processed through binarization and circular edge detection algorithm, and outlier detection and removal technique was used to reject eye-blinking. The pupil size according to the empathy was confirmed for statistical significance with test of normality and independent sample t-test. Statistical analysis results, the pupil size was significantly different between empathy (M ± SD = 0.050 ± 1.817)) and non-empathy (M ± SD = 1.659 ± 1.514) condition (t(92) = -4.629, p = 0.000). The rule of empathy according to the pupil size was defined through discriminant analysis, and the rule was verified (Estimation accuracy: 75%) new 12 subjects (6 men and 6 women, mean age ± SD = 22.84 ± 1.57 years). The method proposed in this study is non-contact camera technology and is expected to be utilized in various virtual reality with smart glasses.

• The Journal of the Korea Contents Association
• /
• v.19 no.6
• /
• pp.214-221
• /
• 2019

Visual perception information acquired through human eyes contains much information on how to view visual stimuli using eye tracking technology, it is possible to acquire and analyze consumer visual information as quantitative data. These measurements can be used to measure emotions that customers feel unconsciously, and they can be directly collected by numerically quantifying the character's search response through eye tracking. In this study, we traced the character's area of interest (AOI) and found that the average of fixation duration, count, average of visit duration, count, and finally the time to first fixation was analyzed. As a result of analysis, it was found that there were many cognitive processing processes on the face than the character's body, and the visual attention was high. The visual attention of attraction factor has also been able to verify that attraction is being presented as an important factor in determining preferences for characters. Based on the results of this study, further studies of more characters will be conducted and quantitative interpretation methods can be used as basic data for character development and factors to be considered in determining character design.

• Journal of Music and Human Behavior
• /
• v.14 no.2
• /
• pp.19-43
• /
• 2017

This study aimed to examine the effects of a reciprocal attention intervention on the joint attention behaviors of children with autism spectrum disorder (ASD) by comparing the music and nonmusic conditions. An alternating treatment design was applied as one of the single subject designs and the reciprocal attention intervention included music and nonmusic conditions implemented alternately within a session. The participants were four children between the ages 4 and 5 years and each participant who participated in 23 intervention sessions that followed the sequence of baseline (3 sessions), treatment (15 sessions), and follow-up (5 sessions). The music condition consisted of structured joint instrument playing, trials for attentional shift, and interactive instrument playing. The nonmusic condition consisted of joint toy play, trials for attentional shift, and turn taking-based play. The occurrence of target behaviors (i. e., joint attention behavior, eye gaze, and joint action) was analyzed across sessions. At pretest and posttest, the Early Social Communication Scale was administered. All participants showed increasing tendency in all target behaviors, but such occurrence was greater in the music condition than in the nonmusic condition. The findings support the use of a reciprocal attention intervention with musical stimuli to effectively improve joint attention in this population.