• KIPS Transactions on Computer and Communication Systems
• /
• v.11 no.10
• /
• pp.373-380
• /
• 2022

This paper presents a deep learning based group synchronization that supports networked immersive interactions between remote users. The goal of group synchronization is to enable all participants to synchronously interact with others for increasing user presence Most previous methods focus on NTP-based clock synchronization to enhance time accuracy. Moving average filters are used to control media playout time on the synchronization server. As an example, the exponentially weighted moving average(EWMA) would be able to track and estimate accurate playout time if the changes in input data are not significant. However it needs more time to be stable for any given change over time due to codec and system loads or fluctuations in network status. To tackle this problem, this work proposes the Deep Group Synchronization(DeepGroupSync), a group synchronization based on deep learning that models important features from the data. This model consists of two Gated Recurrent Unit(GRU) layers and one fully-connected layer, which predicts an optimal playout time by utilizing the sequential playout delays. The experiments are conducted with an existing method that uses the EWMA and the proposed method that uses the DeepGroupSync. The results show that the proposed method are more robust against unpredictable or rapid network condition changes than the existing method.

• Journal of Distribution Science
• /
• v.20 no.11
• /
• pp.121-129
• /
• 2022

Purpose: Artificial intelligence is widely used, particularly in the popular neural network theory called Deep learning. The improvement of computing speed and capability expedited the progress of Deep learning applications. The application of Deep learning in education has various effects and possibilities in creating and managing educational content and services that can replace human cognitive activity. Among Deep learning, Deep fake technology is used to combine and synchronize human faces with voices. This paper will show how to develop e-Learning content videos using those technologies and open-source tools. Research design, data, and methodology: This paper proposes 4 step development process, which is presented step by step on the Google Collab environment with source codes. This technology can produce various video styles. The advantage of this technology is that the characters of the video can be extended to any historical figures, celebrities, or even movie heroes producing immersive videos. Results: Prototypes for each case are also designed, developed, presented, and shared on YouTube for each specific case development. Conclusions: The method and process of creating e-learning video contents from the image, video, and audio files using Deep fake open-source technology was successfully implemented.

• Journal of the Korea Computer Graphics Society
• /
• v.30 no.1
• /
• pp.1-10
• /
• 2024

In immersive virtual environments including mixed reality (MR) and virtual reality (VR), avatars or agents, which are virtual humans, are being studied and applied in various ways as factors that increase users' social presence. Recently, studies are being conducted to apply generative AI as an agent to improve user learning effects or suggest a collaborative environment in an immersive virtual environment. This study proposes a novel method for interface application of a virtual assistant agent (VAA) using OpenAI's ChatGPT in an immersive virtual environment including VR and MR. The proposed method consists of an information agent that responds to user queries and a control agent that controls virtual objects and environments according to user needs. We set up a development environment that integrates the Unity 3D engine, OpenAI, and packages and development tools for user participation in MR and VR. Additionally, we set up a workflow that leads from voice input to the creation of a question query to an answer query, or a control request query to a control script. Based on this, MR and VR experience environments were produced, and experiments to confirm the performance of VAA were divided into response time of information agent and accuracy of control agent. It was confirmed that the interface application of the proposed VAA can increase efficiency in simple and repetitive tasks along with user-friendly features. We present a novel direction for the interface application of an immersive virtual environment through the proposed VAA and clarify the discovered problems and limitations so far.

• Journal of Practical Engineering Education
• /
• v.15 no.2
• /
• pp.373-379
• /
• 2023

With the 4th Industrial Revolution, VR/AR technology is developing, and since COVID-19, the use of VR/AR technology is increasing and is being used in various environments. In particular, AR technology can be used in various educational fields and the perception of VR in the educational field and learning satisfaction, problem-solving ability, and self-directed learning ability when using VR were analyzed. It was analyzed that perception of VR and learning satisfaction had a significant correlation at the significance level of 0.001, and problem-solving competency had a significant correlation between learning satisfaction and self-directed learning ability at the significance level of 0.05.

• International Journal of Advanced Culture Technology
• /
• v.9 no.4
• /
• pp.295-301
• /
• 2021

Digital media technology is gradually developing with the development of convergence quaternary industrial technology and mobile devices. The combination of deep learning and augmented reality can provide more convenient and lively services through the interaction of 3D virtual images with the real world. We combine deep learning-based pose prediction with augmented reality technology. We predict the eight vertices of the bounding box of the object in the image. Using the predicted eight vertices(x,y), eight vertices(x,y,z) of 3D mesh, and the intrinsic parameter of the smartphone camera, we compute the external parameters of the camera through the PnP algorithm. We calculate the distance to the object and the degree of rotation of the object using the external parameter and apply to AR content. Our method provides services in a web environment, making it highly accessible to users and easy to maintain the system. As we provide augmented reality services using consumers' smartphone cameras, we can apply them to various business fields.

• The Journal of Sustainable Design and Educational Environment Research
• /
• v.19 no.1
• /
• pp.13-24
• /
• 2020

The purpose of this study is to analyze and match future teaching-learning methods with learning-space types as customized not only by school grade or grade groups, but also by learning modality. As a result, the following six teaching-learning methods were identified as future teaching-learning methods: flipped learning, deeper learning, collaborative learning, learning through immersive virtual reality, playful learning, and learning through OER(Open Educational Resources). There were also six learning-space types that were identified: playing and discovering space, a making and placement space, a presentation and sharing space, a space for independent study, space as a stage, and space as content(See Tables 8 and 11). Learning-space types and future teaching-learning methods were matched with 22 different types of learning modalities based on the presented degree of utilization by school grade or grade groups(See Table 13).

• International journal of advanced smart convergence
• /
• v.12 no.4
• /
• pp.268-276
• /
• 2023

The representative branch of ICT education is Arduino. However, there are various problems when teaching using Arduino. Arduino requires a complex understanding of hardware and software, and this can be perceived as a difficult course, especially for beginners who are not familiar with programming or electronics. Additionally, the process of connecting the pins of the Arduino board and components must be accurate, and even small mistakes can lead to project failure, which can reduce the learner's concentration and interest in learning Arduino. Existing Arduino learning content consists of text and images in 2D format, which has limitations in increasing student understanding and immersion. Therefore, in this paper analyzes the necessary conditions for sprouting 'growing kidney beans' in the first semester of the fourth grade of elementary school, and builds an automated experimental environment using Arduino. Augmented reality of the pin connection process was designed and produced to solve the difficulties when building an automation system using Arduino. After 3D modeling Arduino and components using 3D Max, animation was set, and augmented reality (AR) content was produced using Unity to provide learners with more intuitive and immersive learning content when learning Arduino. Augmented reality (AR)-based Arduino learning content production is expected to increase educational effects by improving the understanding and immersion of classes in ICT education using Arduino and inducing fun and interest in physical computing coding education.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.18 no.8
• /
• pp.2381-2398
• /
• 2024

The construction of virtual indoor spaces is crucial for the development of metaverses, virtual production, and other 3D content domains. Traditional methods for creating these spaces are often cost-prohibitive and labor-intensive. To address these challenges, we present a pipeline for generating digital twins of real indoor environments from RGB-D camera-scanned data. Our pipeline synergizes space structure estimation, 3D object detection, and the inpainting of missing areas, utilizing deep learning technologies to automate the creation process. Specifically, we apply deep learning models for object recognition and area inpainting, significantly enhancing the accuracy and efficiency of virtual space construction. Our approach minimizes manual labor and reduces costs, paving the way for the creation of metaverse spaces that closely mimic real-world environments. Experimental results demonstrate the effectiveness of our deep learning applications in overcoming traditional obstacles in digital twin creation, offering high-fidelity digital replicas of indoor spaces. This advancement opens for immersive and realistic virtual content creation, showcasing the potential of deep learning in the field of virtual space construction.

• International Journal of Advanced Culture Technology
• /
• v.12 no.3
• /
• pp.419-426
• /
• 2024

Due to the government's announcement of the 2025 policy mandating coding education, there is a growing need for effective coding learning methods in elementary education. However, there are few methods available that can easily help younger students understand coding. While text-based coding and visual block coding methods exist, they have limitations. To address these issues, we propose a block coding learning platform that combines virtual reality (VR) technology with gamification elements. The traditional two dimensional (2D) block coding methods have some limitations, so this platform aims to overcome these by providing an environment where learners can intuitively understand and experience coding in a three dimensional (3D) virtual space. The primary goal is to enhance immersive, learner-centered experiences and improve creative problem-solving skills and computational thinking. This study proposes an experimental approach to demonstrate the effectiveness of a learning platform that combines VR technology with block coding. Furthermore, we expect that the VR-based platform will significantly contribute to improving the quality of education and promoting self-directed learning among students.

• Journal of the Ergonomics Society of Korea
• /
• v.36 no.5
• /
• pp.385-394
• /
• 2017

Objective: The aim of this study is to investigate how to design effective virtual reality-based training (i.e., virtual training) in maritime safety and to present methods for enhancing interface fidelity by employing immersive interaction and 3D user interface (UI) design. Background: Emerging virtual reality technologies and hardware enable to provide immersive experiences to individuals. There is also a theory that the improvement of fidelity can improve the training efficiency. Such a sense of immersion can be utilized as an element for realizing effective training in the virtual space. Method: As an immersive interaction, we implemented gesture-based interaction using leap motion and Myo armband type sensors. Hand gestures captured from both sensors are used to interact with the virtual appliance in the scenario. The proposed 3D UI design is employed to visualize appropriate information for tasks in training. Results: A usability study to evaluate the effectiveness of the proposed method has been carried out. As a result, the usability test of satisfaction, intuitiveness of UI, ease of procedure learning, and equipment understanding showed that virtual training-based exercise was superior to existing training. These improvements were also independent of the type of input devices for virtual training. Conclusion: We have shown through experiments that the proposed interaction design results are more efficient interactions than the existing training method. The improvement of interface fidelity through intuitive and immediate feedback on the input device and the training information improve user satisfaction with the system, as well as training efficiency. Application: Design methods for an effective virtual training system can be applied to other areas by which trainees are required to do sophisticated job with their hands.