• KSII Transactions on Internet and Information Systems (TIIS)
• /
• 제13권8호
• /
• pp.4026-4040
• /
• 2019

In this paper, we present an approach for detecting slide transitions in lecture videos by introducing the spatio-temporal residual networks. Given a lecture video which records the digital slides, the speaker, and the audience by multiple cameras, our goal is to find keyframes where slide content changes. Since temporal dependency among video frames is important for detecting slide changes, 3D Convolutional Networks has been regarded as an efficient approach to learn the spatio-temporal features in videos. However, 3D ConvNet will cost much training time and need lots of memory. Hence, we utilize ResNet to ease the training of network, which is easy to optimize. Consequently, we present a novel ConvNet architecture based on 3D ConvNet and ResNet for slide transition detection in lecture videos. Experimental results show that the proposed novel ConvNet architecture achieves the better accuracy than other slide progression detection approaches.

• 한국컴퓨터정보학회논문지
• /
• 제26권3호
• /
• pp.35-42
• /
• 2021

본 연구에서는 3차원 RGB-D Xtion2 카메라를 이용하여 보행자의 골격좌표를 추출한 결과를 바탕으로 동적인 특성(속도, 가속도)을 함께 고려하여 딥러닝 모델을 통해 사람을 인식하는 방법을 제안한다. 본 논문의 핵심목표는 RGB-D 카메라로 손쉽게 좌표를 추출하고 새롭게 생성한 동적인 특성을 기반으로 자체 고안한 1차원 합성곱 신경망 분류기 모델(1D-ConvNet)을 통해 자동으로 보행 패턴을 파악하는 것이다. 1D-ConvNet의 인식 정확도와 동적인 특성이 정확도에 미치는 영향을 알아보기 위한 실험을 수행하였다. 정확도는 F1 Score를 기준으로 측정하였고, 동적인 특성을 고려한 분류기 모델(JCSpeed)과 고려하지 않은 분류기 모델(JC)의 정확도 비교를 통해 영향력을 측정하였다. 그 결과 동적인 특성을 고려한 경우의 분류기 모델이 그렇지 않은 경우보다 F1 Score가 약 8% 높게 나타났다.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• 제12권11호
• /
• pp.5555-5567
• /
• 2018

State-of-art instance segmentation networks are successful at generating 2D segmentation mask for region proposals with highest classification score, yet 3D object segmentation task is limited to geocentric embedding or detector of Sliding Shapes. To this end, we propose an amodal 3D instance segmentation network called A3IS-CNN, which extends the detector of Deep Sliding Shapes to amodal 3D instance segmentation by adding a new branch of 3D ConvNet called A3IS-branch. The A3IS-branch which takes 3D amodal ROI as input and 3D semantic instances as output is a fully convolution network(FCN) sharing convolutional layers with existing 3d RPN which takes 3D scene as input and 3D amodal proposals as output. For two branches share computation with each other, our 3D instance segmentation network adds only a small overhead of 0.25 fps to Deep Sliding Shapes, trading off accurate detection and point-to-point segmentation of instances. Experiments show that our 3D instance segmentation network achieves at least 10% to 50% improvement over the state-of-art network in running time, and outperforms the state-of-art 3D detectors by at least 16.1 AP.

• Journal of Platform Technology
• /
• 제12권1호
• /
• pp.3-11
• /
• 2024

Deaf people who use sign language as their first language sometimes have difficulty communicating because they do not know spoken Korean. Deaf people are also members of society, so we must support to create a society where everyone can live together. In this paper, we present a method to increase the recognition rate of Korean sign language using a CNN model. When the original image was used as input to the CNN model, the accuracy was 0.96, and when the image corresponding to the skin area in the YCbCr color space was used as input, the accuracy was 0.72. It was confirmed that inserting the original image itself would lead to better results. In other studies, the accuracy of the combined Conv1d and LSTM model was 0.92, and the accuracy of the AlexNet model was 0.92. The CNN model proposed in this paper is 0.96 and is proven to be helpful in recognizing Korean sign language.