• Journal of Convergence for Information Technology
• /
• v.7 no.6
• /
• pp.245-251
• /
• 2017

This presents an English conversation training system with which learners train their conversation skills in English, which makes them converse with native speaker characters in a virtual reality environment with voice. The proposed system allows the learners to talk with multiple native speaker characters in varous scenarios in the virtual reality environment. It recongizes voices spoken by the learners and generates voices by a speech synthesis method. The interaction with characters in the virtual reality environment in voice makes the learners immerged in the conversation situations. The scoring system which evaluates the learner's pronunciation provides the positive feedback for the learners to get engaged in the learning context.

• Journal of the Korea Convergence Society
• /
• v.10 no.11
• /
• pp.55-61
• /
• 2019

In order to realize foreign language education, various existing educational media have been provided, but there are disadvantages in that the cost of the parish and the media program is high and the real-time responsiveness is poor. In this paper, we propose an artificial intelligence English conversation system based on VR and speech recognition. We used Google CardBoard VR and Google Speech API to build the system and developed artificial intelligence algorithms for providing virtual reality environment and talking. In the proposed speech recognition server system, the sentences spoken by the user can be divided into word units and compared with the data words stored in the database to provide the highest probability. Users can communicate with and respond to people in virtual reality. The function provided by the conversation is independent of the contextual conversations and themes, and the conversations with the AI assistant are implemented in real time so that the user system can be checked in real time. It is expected to contribute to the expansion of virtual education contents service related to the Fourth Industrial Revolution through the system combining the virtual reality and the voice recognition function proposed in this paper.

• Journal of the Korea Convergence Society
• /
• v.11 no.1
• /
• pp.119-130
• /
• 2020

This study developed an English conversation learning program in immersive virtual reality (VR) environments and compared its effects with non-immersive VR environment using a computer monitor. The effects of the program was assessed using language anxiety and learning achievement. The results indicated that students' language anxiety decreased significantly after learning English conversation in VR environment, but there was no difference between immersive and non-immersive VR. The two VR conditions also produced similar learning outcomes. Future research on immersive VR need to address cyber sickness problems and develop effective learning contents in order to realize its potential for learning.

• International journal of advanced smart convergence
• /
• v.11 no.4
• /
• pp.120-128
• /
• 2022

While online social interactions have been more prevalent with the increased popularity of Metaverse platforms, little has been studied the effects of facial expressions in virtual reality (VR), which is known to play a key role in social contexts. To understand the importance of presenting facial expressions of a virtual avatar under different contexts, we conducted a user study with 24 participants where they were asked to have a conversation and play a charades game with an avatar with and without facial expressions. The results show that participants tend to gaze at the face region for the majority of the time when having a conversation or trying to guess emotion-related keywords when playing charades regardless of the presence of facial expressions. Yet, we confirmed that participants prefer to see facial expressions in virtual reality as well as in real-world scenarios as it helps them to better understand the contexts and to have more immersive and focused experiences.

• The Journal of the Korea Contents Association
• /
• v.20 no.2
• /
• pp.581-593
• /
• 2020

The purpose of this study was to examine the eye-fixation times on different parts of a student avatar and the virtual presence with two scenarios in the virtual reality-based teaching simulation. This study was to identify user attention while he or she is interacting with a student avatar. By examining where a user is gazing during a conversation with the avatar, we have a better understanding of non-verbal communication. For this study, forty-five college students (21 females and 24 males) participated in the experiment. They had a conversation with a student avatar in a virtual reality-based teaching simulation. The participants had verbal interactions with the student avatar with two scenarios. While they were having a conversation with the virtual character in the teaching simulation, their eye-movements were collected through a head-mounted display with an eye-tracking function embedded. The results revealed that there were significant differences in eye-fixation times. Participants gazed a longer time on facial expression than any other area. The fixation time on the facial expression was more prolonged than on gestures (F=3.75, p<.05). However, the virtual presence was not significantly different in two scenario levels. This result suggested that users focus on the face more than the gesture when they emotionally interact with the virtual character.

• The Journal of Korean Association of Computer Education
• /
• v.3 no.1
• /
• pp.161-168
• /
• 2000

In the 21st century, we need to educate students who can perform their learning activity by themselves through free investigation and gathering information instead of providing a piece of knowledge to them directly. It is suggested that we should use Internet to bring up students who have the ability to study self-directedly. In this study, we designed a virtual English conversation class and implemented it using Web and DBMS to improve the students' self-directed learning ability who study English conversation in high school.

• The Journal of the Korea Contents Association
• /
• v.21 no.1
• /
• pp.321-332
• /
• 2021

This study developed an English conversation learning program using virtual reality(VR) and mobile devices. Participants learned and practiced English conversational patterns in immersive virtual reality and mobile conditions. In the program, participants learned and practiced nine conversational patterns with virtual characters in four steps. Language anxiety and conversational fluency were measured to examine the effects of this program. Language anxiety questionnaire was administered before and after the experiment. The results showed that language anxiety was significantly reduced after learning in both conditions, and the reduction waa significantly greater in the immersive condition. Conversational fluency was assessed based on the changes in the length, appropriateness, and accuracy of the responses before and after participants learned and practiced conversational episodes. The results showed that the length, appropriateness, and accuracy of the responses were improved in both conditions after learning. The response length was significantly longer in the immersive VR conditions. These results suggest that immersive VR can be an effective tool to enhance English conversational abilities.

• Journal of Korea Multimedia Society
• /
• v.23 no.10
• /
• pp.1296-1306
• /
• 2020

Childhood verbal communication impacts children's language skills and has a positive effect as partners use more vocabulary. But reduction in family time, caused by lowered age for private education and so on, has reduced the chance for children to speak with partners who have a proficient language skill. This vacancy was naturally occupied by the media, which has become one of the cornerstones of the growth of kids' contents. Kids contents are making various attempts to expand the breadth of services. But most contents still focus on unilateral visual information delivery yet, so there is a limit to satisfy the vacancy of conversation partners. Therefore this paper suggests an ECA(Embodied conversational agent) to induce children's spoken conversation using a virtual character frequently used in kids contents. This system is implemented by the voice bot and agent model produced using an IBM assistant and Unity. As a result of using ECA for 66 children of 5-9 years old, it showed meaningful results in terms of induction of verbal communication.

• Journal of the Korea Computer Graphics Society
• /
• v.29 no.4
• /
• pp.1-6
• /
• 2023

To enhance immersive experiences for metaverse environements, background music is often used. However, the background music is mostly pre-matched and repeated which might occur a distractive experience to users as it does not align well with rapidly changing user-interactive contents. Thus, we implemented a system to provide a more immersive metaverse conversation experience by 1) developing a regression neural network that extracts emotions from an utterance using KEMDy20, the Korean multimodal emotion dataset 2) selecting music corresponding to the extracted emotions from an utterance by the DEAM dataset where music is tagged with arousal-valence levels 3) combining it with a virtual space where users can have a real-time conversation with avatars.

• The Journal of the Korea Contents Association
• /
• v.5 no.4
• /
• pp.122-132
• /
• 2005

CVE(Collaborative Virtual Environment) is the virtually shared area where people cannot come together physically, but wish to discuss, collaborate on, or even dispute certain matters. In CVEs, in habitants are usually represented by humanoid embodiments, generally referred to as avatars. But most current graphical CVE systems fail to reflect the natural relationship between the avatar's gesture and the conversation that is taking place. More than 65% of the information exchanged during a person to person conversation is carried on the nonverbal band. Therefore, it is expected to be beneficial to provide such communication channels in CVEs in some way. To address this issue, this study proposes a scheme to represent avatar's gestures that can support the CVE users' communication. In the first level, this study classifies the non-verbal communication forms that can be applicable to avatar gesture design. In the second level, this study categorizes the body language according to the types of interaction with verbal language. And in the third level, this study examines gestures with relevant verbal expressions according to the body parts-from head to feet. One bodily gesture can be analyzed in the description of gesture representation, the meaning of gesture and the possible expressions, which can be used in gestural situation.