• Journal of Digital Contents Society
• /
• v.18 no.3
• /
• pp.559-565
• /
• 2017

A lot of studies have been conducted to recognize the motion state or behavior of the user using the acceleration sensor built in the smartphone. In this paper, we applied the neural networks to the 3-axis acceleration information of smartphone to study human behavior. There are performance issues in applying time series data to neural networks. We proposed a multi-temporal neural networks which have trained three neural networks with different time windows for feature extraction and uses the output of these neural networks as input to the new neural network. The proposed method showed better performance than other methods like SVM, AdaBoot and IBk classifier for real acceleration data.

• Journal of Broadcast Engineering
• /
• v.24 no.4
• /
• pp.553-563
• /
• 2019

As the number of videos uploaded on live streaming platforms rapidly increases, the demand for providing highlight videos is increasing to promote viewer experiences. In this paper, we present novel methods for predicting highlights using chat logs and audio data in videos. The proposed models employ bi-directional LSTMs to understand the contextual flow of a video. We also propose to use the features over various time-intervals to understand the mid-to-long term flows. The proposed Our methods are demonstrated on e-Sports and baseball videos collected from personal broadcasting platforms such as Twitch and Kakao TV. The results show that the information from multiple time-intervals is useful in predicting video highlights.

• Journal of Broadcast Engineering
• /
• v.25 no.2
• /
• pp.143-150
• /
• 2020

Huge amounts of contents are being uploaded every day on various streaming platforms. Among those videos, game and sports videos account for a great portion. The broadcasting companies sometimes create and provide highlight videos. However, these tasks are time-consuming and costly. In this paper, we propose models that automatically predict highlights in games and sports matches. While most previous approaches use visual information exclusively, our models use both audio and visual information, and present a way to understand short term and long term flows of videos. We also describe models that combine GAN to find better highlight features. The proposed models are evaluated on e-sports and baseball videos.