• Journal of Korea Multimedia Society
• /
• v.23 no.12
• /
• pp.1464-1475
• /
• 2020

Due to the spread of COVID-19, the online lecture has become more prevalent. However, it was found that a lot of students and professors are experiencing lack of communication. This study is therefore designed to improve interactive communication between professors and students in real-time online lectures. To do so, we explore deep learning approaches for automatic recognition of students' facial expressions and classification of their understanding into 3 classes (Understand / Neutral / Not Understand). We use 'BlazeFace' model for face detection and 'ResNet-GRU' model for facial expression recognition (FER). We name this entire process 'Degree of Understanding (DoU)' algorithm. DoU algorithm can analyze a multitude of students collectively and present the result in visualized statistics. To our knowledge, this study has great significance in that this is the first study offers the statistics of understanding in lectures using FER. As a result, the algorithm achieved rapid speed of 0.098sec/frame with high accuracy of 94.3% in CPU environment, demonstrating the potential to be applied to real-time online lectures. DoU Algorithm can be extended to various fields where facial expressions play important roles in communications such as interactions with hearing impaired people.

• Journal of the Korea Institute of Information and Communication Engineering
• /
• v.25 no.1
• /
• pp.44-49
• /
• 2021

Recently, due to COVID-19, studies have been popularly worked to apply neural network to mask wearing automatic detection system. For applying neural networks, the 1-stage detection or 2-stage detection methods are used, and if data are not sufficiently collected, the pretrained neural network models are studied by applying fine-tuning techniques. In this paper, the system is consisted of 2-stage detection method that contain MTCNN model for face recognition and ResNet model for mask detection. The mask detector was experimented by applying five ResNet models to improve accuracy and fps in various environments. Training data used 17,217 images that collected using web crawler, and for inference, we used 1,913 images and two one-minute videos respectively. The experiment showed a high accuracy of 96.39% for images and 92.98% for video, and the speed of inference for video was 10.78fps.

• Journal of the Korea Institute of Information and Communication Engineering
• /
• v.26 no.3
• /
• pp.389-395
• /
• 2022

Because of improvement of deep learning techniques, deep learning using computer vision such as classification, detection and segmentation has also been used widely at many fields. Expecially, automatic driving is one of the major fields that applies computer vision systems. Also there are a lot of works and researches to combine multiple tasks in a single network. In this study, we propose the network that predicts the individual depth of pedestrians and vehicles. Proposed model is constructed based on YOLOv3 for object detection and Monodepth for depth estimation, and it process object detection and depth estimation consequently using encoder and decoder based on hard parameter sharing. We also used attention module to improve the accuracy of both object detection and depth estimation. Depth is predicted with monocular image, and is trained using self-supervised training method.

• Journal of Biomedical Engineering Research
• /
• v.43 no.6
• /
• pp.424-433
• /
• 2022

As the proportion of non-contact telemedicine increases and the number of electrocardiogram (ECG) data measured using portable ECG monitors increases, the demand for automatic algorithms that can precisely analyze vast amounts of ECG is increasing. Since the P, QRS, and T waves of the ECG have different shapes depending on the location of electrodes or individual characteristics and often have similar frequency components or amplitudes, it is difficult to distinguish P, QRS and T waves and measure each parameter. In order to measure the widths, intervals and areas of P, QRS, and T waves, a new algorithm that recognizes the start and end points of each wave and automatically measures the time differences and amplitudes between each point is required. In this study, the start and end points of the P, QRS, and T waves were measured using six Deep Neural Networks (DNN) that recognize the start and end points of each wave. Then, by synthesizing the results of all DNNs, 12 parameters for ECG characteristics for each heartbeat were obtained. In the ECG waveform of 10 subjects provided by Physionet, 12 parameters were measured for each of 660 heartbeats, and the 12 parameters measured for each heartbeat well represented the characteristics of the ECG, so it was possible to distinguish them from other subjects' parameters. When the ECG data of 10 subjects were combined into one file and analyzed with the suggested algorithm, 10 types of ECG waveform were observed, and two types of ECG waveform were simultaneously observed in 5 subjects, however, it was not observed that one person had more than two types.

• The Korea Journal of Herbology
• /
• v.34 no.3
• /
• pp.37-44
• /
• 2019

Objectives : Acne is a common disease that affects more than 70% of adolescents. Acne patients have a poor quality of life compared to patients with other skin diseases. We tried to demonstrate the effectiveness of acne treatment using cleansing soap which is easily used in daily life. Methods : We selected 20 students with acne symptoms on their facial skin. We made herbal Cp (Cold process) soaps using Seosiokyongsan, Kyungohkgo, Hwangryunhaedoktang and Baeksoooh and distributed them to experiment participants. We let them wash their face in the morning and evening for 6 weeks using herbal Cp soap. Prior to the experiment, their skin condition was checked and assessed using A-ONE Smart One-Click Automatic Facial Diagnosis System three times at 3-week intervals. Acne status was classified into 6 stages according to KAGS and acne status was also measured 3 times in total. After the experiment, the changes of skin were analyzed through facial analysis test. Results : Based on the KAGS classification, the condition of acne has improved as a whole. The state of moisture was gradually increased and the state of skin oil was significantly decreased after 6 weeks of using soap compared to before using soap. Conclusions : Cp soaps made from four kinds of herbal medicine are believed to improve the condition of acne by increasing the moisture of the facial skin and decreasing the skin oil content.

• International Journal of Computer Science & Network Security
• /
• v.22 no.6
• /
• pp.230-240
• /
• 2022

Sharing of online videos via internet is an emerging and important concept in different types of applications like surveillance and video mobile search in different web related applications. So there is need to manage personalized web video retrieval system necessary to explore relevant videos and it helps to peoples who are searching for efficient video relates to specific big data content. To evaluate this process, attributes/features with reduction of dimensionality are computed from videos to explore discriminative aspects of scene in video based on shape, histogram, and texture, annotation of object, co-ordination, color and contour data. Dimensionality reduction is mainly depends on extraction of feature and selection of feature in multi labeled data retrieval from multimedia related data. Many of the researchers are implemented different techniques/approaches to reduce dimensionality based on visual features of video data. But all the techniques have disadvantages and advantages in reduction of dimensionality with advanced features in video retrieval. In this research, we present a Novel Intent based Dimension Reduction Semi-Supervised Learning Approach (NIDRSLA) that examine the reduction of dimensionality with explore exact and fast video retrieval based on different visual features. For dimensionality reduction, NIDRSLA learns the matrix of projection by increasing the dependence between enlarged data and projected space features. Proposed approach also addressed the aforementioned issue (i.e. Segmentation of video with frame selection using low level features and high level features) with efficient object annotation for video representation. Experiments performed on synthetic data set, it demonstrate the efficiency of proposed approach with traditional state-of-the-art video retrieval methodologies.

• Journal of Korea Multimedia Society
• /
• v.25 no.5
• /
• pp.748-756
• /
• 2022

Thanks to the competition of AlphaGo and Sedol Lee, machine learning has received world-wide attention and huge investments. The performance improvement of computing devices greatly contributed to big data processing and the development of neural networks. Artificial intelligence not only imitates human beings in many fields, but also seems to be better than human capabilities. Although humans' creation is still considered to be better and higher, several artificial intelligences continue to challenge human creativity. The quality of some creative outcomes by AI is as good as the real ones produced by human beings. Sometimes they are not distinguishable, because the neural network has the competence to learn the common features contained in big data and copy them. In order to confirm whether artificial intelligence can express the inherent characteristics of different arts, this paper proposes a new neural network model called Humming. It is an experimental model that combines vgg16, which extracts image features, and DeepJ's architecture, which excels in creating various genres of music. A dataset produced by our experiment shows meaningful and valid results. Different results, however, are produced when the amount of data is increased. The neural network produced a similar pattern of music even though it was a different classification of images, which was not what we were aiming for. However, these new attempts may have explicit significance as a starting point for feature transfer that will be further studied.

• Journal of Korean Society of Water and Wastewater
• /
• v.36 no.3
• /
• pp.149-165
• /
• 2022

The water consumption data of individual consumers must be analyzed and forecast to establish an effective water demand management plan. A k-mean cluster model that can monitor water use characteristics based on hourly water consumption data measured using automated meter reading devices and demographic factors is developed in this study. In addition, the quantification model that can estimate the daily water consumption is developed. K-mean cluster analysis based on the four clusters shows that the average silhouette coefficient is 0.63, also the silhouette coefficients of each cluster exceed 0.60, thereby verifying the high reliability of the cluster analysis. Furthermore, the clusters are clearly classified based on water usage and water usage patterns. The correlation coefficients of four quantification models for estimating water consumption exceed 0.74, confirming that the models can accurately simulate the investigated demographic data. The statistical significance of the models is considered reasonable, hence, they are applicable to the actual field. Because the use of automated smart water meters has become increasingly popular in recent year, water consumption has been metered remotely in many areas. The proposed methodology and the results obtained in this study are expected to facilitate improvements in the usability of smart water meters in the future.

• Journal of the Institute of Convergence Signal Processing
• /
• v.23 no.2
• /
• pp.62-69
• /
• 2022

An electrocardiogram (ECG) is one of the most important biosignals, and in particular, continuous ECG monitoring is very important in patients with arrhythmia. There are many different types of arrhythmia (sinus node, sinus tachycardia, atrial premature beat (APB), and ventricular fibrillation) depending on the cause, and continuous ECG monitoring during daily life is very important for early diagnosis of arrhythmias and setting treatment directions. The ECG signal of arrhythmia patients is very unstable, and it is difficult to detect the R-peak point, which is a key feature for automatic arrhythmias detection. In this study, we develped a continuous measuring Holter ECG monitoring device and software for analysis and confirmed the utility of R-peak of the ECG signal with MIT-BIH arrhythmia database. In future studies, it needs the validation of algorithms and clinical data for morphological classification and prediction of arrhythmias due to various etiologies.

• Journal of Drive and Control
• /
• v.20 no.2
• /
• pp.15-23
• /
• 2023

In this study, two-dimensional location of crops for auto weeding was detected using deep learning. To construct a dataset for soybean detection, an image-capturing system was developed using a mono camera and single-board computer and the system was mounted on a weeding robot to collect soybean images. A dataset was constructed by extracting RoI (region of interest) from the raw image and each sample was labeled with soybean and the background for classification learning. The deep learning model consisted of four convolutional layers and was trained with a weakly supervised learning method that can provide object localization only using image-level labeling. Localization of the soybean area can be visualized via CAM and the two-dimensional position of the soybean was estimated by clustering the pixels associated with the soybean area and transforming the pixel coordinates to world coordinates. The actual position, which is determined manually as pixel coordinates in the image was evaluated and performances were 6.6(X-axis), 5.1(Y-axis) and 1.2(X-axis), 2.2(Y-axis) for MSE and RMSE about world coordinates, respectively. From the results, we confirmed that the center position of the soybean area derived through deep learning was sufficient for use in automatic weeding systems.