• 한국정보통신학회논문지
• /
• 제26권6호
• /
• pp.813-826
• /
• 2022

카메라를 이용한 영상 처리와 그에 따른 인공지능 기술의 발달로 다양한 분야의 기술이 발전하기 시작했다. 하지만 보드가 가벼울수록 연산이 많이 필요한 영상 처리 알고리즘을 구현하기 힘들다. 본 논문에서는 경량 임베디드 보드에서 물체 인식 알고리즘을 위한 딥러닝을 사용하는 방법을 제안한다. 비교적 적은 양의 계산으로 segmentation을 처리하는 딥러닝 알고리즘을 사용하여 ROI(Region of Interest)를 결정할 수 있다. 영역을 마스킹한 후, 더 정확한 딥러닝 알고리즘을 사용해 물체 감지를 할 수 있다. Python에서 입력 이미지를 처리하기 위해 OpenCV를 사용했고 ENet과 YOLO(You Only Look Once)를 사용하여 이미지를 처리했다. 이 알고리즘을 실행함으로써 평균 오차가 절반으로 감소해 정확한 객체 검출을 처리할 수 있고 경량 임베디드 보드에서 실시간으로 객체 인식을 실행할 수 있다. 이 연구는 자율주행과 IoT에서 저가격 경량화된 응용에 활용될 수 있을 것으로 기대된다.

• 융합신호처리학회논문지
• /
• 제22권4호
• /
• pp.156-163
• /
• 2021

어텐션(Attention) 메커니즘은 딥러닝 기술을 활용한 다양한 컴퓨터 비전 시스템에서 활용되고 있으며, 초해상화(Super-resolution)를 위한 딥러닝 모델에도 어텐션 메커니즘을 적용하고 있다. 하지만 어텐션 메커니즘이 적용된 대부분의 초해상화 기법들은 Real 영상의 초해상화에만 초점을 맞추어서 연구되어, 어텐션 메커니즘을 적용한 초해상화가 CG나 VR 영상 초해상화에도 유효한지는 알기 어렵다. 본 논문에서는 최근에 제안된 어텐션 메커니즘 모듈인 BAM(Balanced Attention Mechanism) 모듈을 12개의 초해상화 딥러닝 모델에 적용한 후, CG나 VR 영상에서도 성능 향상 효과를 보이는지 확인하는 실험을 진행하였다. 실험 결과, BAM 모듈은 제한적으로 CG나 VR 영상의 초해상화 성능 향상에 기여하였으며, 데이터 특징과 크기, 그리고 네트워크 종류에 따라 성능 향상도가 달라진다는 것을 확인할 수 있었다.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• 제15권11호
• /
• pp.4065-4083
• /
• 2021

Super-resolution can improve the clarity of low-resolution (LR) images, which can increase the accuracy of high-level compute vision tasks. Portable devices have low computing power and storage performance. Large-scale neural network super-resolution methods are not suitable for portable devices. In order to save the computational cost and the number of parameters, Lightweight image processing method can improve the processing speed of portable devices. Therefore, we propose the Enhanced Information Multiple Distillation Network (EIMDN) to adapt lower delay and cost. The EIMDN takes feedback mechanism as the framework and obtains low level features through high level features. Further, we replace the feature extraction convolution operation in Information Multiple Distillation Block (IMDB), with Ghost module, and propose the Enhanced Information Multiple Distillation Block (EIMDB) to reduce the amount of calculation and the number of parameters. Finally, coordinate attention (CA) is used at the end of IMDB and EIMDB to enhance the important information extraction from Spaces and channels. Experimental results show that our proposed can achieve convergence faster with fewer parameters and computation, compared with other lightweight super-resolution methods. Under the condition of higher peak signal-to-noise ratio (PSNR) and higher structural similarity (SSIM), the performance of network reconstruction image texture and target contour is significantly improved.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• 제16권2호
• /
• pp.365-380
• /
• 2022

The target detection algorithm based on supervised learning is the current mainstream algorithm for target detection. A high-quality dataset is the prerequisite for the target detection algorithm to obtain good detection performance. The larger the number and quality of the dataset, the stronger the generalization ability of the model, that is, the dataset determines the upper limit of the model learning. The convolutional neural network optimizes the network parameters in a strong supervision method. The error is calculated by comparing the predicted frame with the manually labeled real frame, and then the error is passed into the network for continuous optimization. Strongly supervised learning mainly relies on a large number of images as models for continuous learning, so the number and quality of images directly affect the results of learning. This paper proposes a dataset STAR-24K (meaning a dataset for Space TArget Recognition with more than 24,000 images) for detecting common targets in space. Since there is currently no publicly available dataset for space target detection, we extracted some pictures from a series of channels such as pictures and videos released by the official websites of NASA (National Aeronautics and Space Administration) and ESA (The European Space Agency) and expanded them to 24,451 pictures. We evaluate popular object detection algorithms to build a benchmark. Our STAR-24K dataset is publicly available at https://github.com/Zzz-zcy/STAR-24K.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• 제16권12호
• /
• pp.3976-3990
• /
• 2022

Currently, cross-attention is widely used in Siamese trackers to replace traditional correlation operations for feature fusion between template and search region. The former can establish a similar relationship between the target and the search region better than the latter for robust visual object tracking. But existing trackers using cross-attention only focus on rich semantic information of high-level features, while ignoring the appearance information contained in low-level features, which makes trackers vulnerable to interference from similar objects. In this paper, we propose a Multi-level Cross-attention Siamese network(MCSiam) to aggregate the semantic information and appearance information at the same time. Specifically, a multi-level cross-attention module is designed to fuse the multi-layer features extracted from the backbone, which integrate different levels of the template and search region features, so that the rich appearance information and semantic information can be used to carry out the tracking task simultaneously. In addition, before cross-attention, a target-aware module is introduced to enhance the target feature and alleviate interference, which makes the multi-level cross-attention module more efficient to fuse the information of the target and the search region. We test the MCSiam on four tracking benchmarks and the result show that the proposed tracker achieves comparable performance to the state-of-the-art trackers.

• International Journal of Contents
• /
• 제18권2호
• /
• pp.47-57
• /
• 2022

e-Learning content can be defined as digital content to achieve educational goals. Since it is an educational material that can be distributed in offline, online, and mobile environments, it is important to create content that meets the learner's education environment and educational goals. In particular, if the learner is a public official, the vision, philosophy, and characteristics of each local government must reflect. As non-face-to-face online education expands further due to the COVID-19 pandemic, local governments that have relied on onsite education in the past urgently require developing strong basic competency education and special task competency content that reflect regional characteristics. Such e-learning content, however, hardly exists and the ability to independently develop them is also insufficient. In this circumstance, this case study describes the process of self-production of e-learning content suitable for Busan's characteristics by the Human Resource Development (HRD) Institute of Busan City, a local government. The field of instructional design and instructional technology is always evolving and growing by blending technological innovation into instructional platform design and adapting to the changes in society. Busan HRD Institute (BHI), therefore, tried to implement blended learning by developing content that reflected the recent trend of micro-learning in e-learning through a detailed analysis. For this, an e-learning content developer with certain requirements was selected and contracted, and the process of developing content through a collaboration between the client and developer was described in this study according to the ADDIE model of Instructional Systems Development (ISD).

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• 제16권8호
• /
• pp.2688-2703
• /
• 2022

Research on the advanced detection of harmful objects in airport cargo for passenger safety against terrorism has increased recently. However, because associated studies are primarily focused on the detection of relatively large objects, research on the detection of small objects is lacking, and the detection performance for small objects has remained considerably low. Here, we verified the limitations of existing research on object detection and developed a new model called the Small Hazardous Object detection enhanced and reconstructed Model based on the You Only Look Once version 5 (YOLOv5) algorithm to overcome these limitations. We also examined the performance of the proposed model through different experiments based on YOLOv5, a recently launched object detection model. The detection performance of our model was found to be enhanced by 0.3 in terms of the mean average precision (mAP) index and 1.1 in terms of mAP (.5:.95) with respect to the YOLOv5 model. The proposed model is especially useful for the detection of small objects of different types in overlapping environments where objects of different sizes are densely packed. The contributions of the study are reconstructed layers for the Small Hazardous Object detection enhanced and reconstructed Model based on YOLOv5 and the non-requirement of data preprocessing for immediate industrial application without any performance degradation.

• Smart Structures and Systems
• /
• 제31권4호
• /
• pp.421-436
• /
• 2023

This paper presents a vibration displacement measurement and damage identification method for a space truss structure from its vibration videos. Features from Accelerated Segment Test (FAST) algorithm is combined with adaptive threshold strategy to detect the feature points of high quality within the Region of Interest (ROI), around each node of the truss structure. Then these points are tracked by Kanade-Lucas-Tomasi (KLT) algorithm along the video frame sequences to obtain the vibration displacement time histories. For some cases with the image plane not parallel to the truss structural plane, the scale factors cannot be applied directly. Therefore, these videos are processed with homography transformation. After scale factor adaptation, tracking results are expressed in physical units and compared with ground truth data. The main operational frequencies and the corresponding mode shapes are identified by using Subspace Stochastic Identification (SSI) from the obtained vibration displacement responses and compared with ground truth data. Structural damages are quantified by elemental stiffness reductions. A Bayesian inference-based objective function is constructed based on natural frequencies to identify the damage by model updating. The Success-History based Adaptive Differential Evolution with Linear Population Size Reduction (L-SHADE) is applied to minimise the objective function by tuning the damage parameter of each element. The locations and severities of damage in each case are then identified. The accuracy and effectiveness are verified by comparison of the identified results with the ground truth data.

• Smart Structures and Systems
• /
• 제31권4호
• /
• pp.335-349
• /
• 2023

Bridges constantly undergo deterioration and damage, the most common ones being concrete damage and exposed rebar. Periodic inspection of bridges to identify damages can aid in their quick remediation. Likewise, identifying components can provide context for damage assessment and help gauge a bridge's state of interaction with its surroundings. Current inspection techniques rely on manual site visits, which can be time-consuming and costly. More recently, robotic inspection assisted by autonomous data analytics based on Computer Vision (CV) and Artificial Intelligence (AI) has been viewed as a suitable alternative to manual inspection because of its efficiency and accuracy. To aid research in this avenue, this study performs a comparative assessment of different architectures, loss functions, and ensembling strategies for the autonomous segmentation of bridge components and damages. The experiments lead to several interesting discoveries. Nested Reg-UNet architecture is found to outperform five other state-of-the-art architectures in both damage and component segmentation tasks. The architecture is built by combining a Nested UNet style dense configuration with a pretrained RegNet encoder. In terms of the mean Intersection over Union (mIoU) metric, the Nested Reg-UNet architecture provides an improvement of 2.86% on the damage segmentation task and 1.66% on the component segmentation task compared to the state-of-the-art UNet architecture. Furthermore, it is demonstrated that incorporating the Lovasz-Softmax loss function to counter class imbalance can boost performance by 3.44% in the component segmentation task over the most employed alternative, weighted Cross Entropy (wCE). Finally, weighted softmax ensembling is found to be quite effective when used synchronously with the Nested Reg-UNet architecture by providing mIoU improvement of 0.74% in the component segmentation task and 1.14% in the damage segmentation task over a single-architecture baseline. Overall, the best mIoU of 92.50% for the component segmentation task and 84.19% for the damage segmentation task validate the feasibility of these techniques for autonomous bridge component and damage segmentation using RGB images.

• Smart Structures and Systems
• /
• 제31권4호
• /
• pp.351-363
• /
• 2023

Visual structural inspections are an inseparable part of post-earthquake damage assessments. With unmanned aerial vehicles (UAVs) establishing a new frontier in visual inspections, there are major computational challenges in processing the collected massive amounts of high-resolution visual data. We propose twin deep learning models that can provide accurate high-resolution structural components and damage segmentation masks efficiently. The traditional approach to cope with high memory computational demands is to either uniformly downsample the raw images at the price of losing fine local details or cropping smaller parts of the images leading to a loss of global contextual information. Therefore, our twin models comprising Trainable Resizing for high-resolution Segmentation Network (TRS-Net) and DmgFormer approaches the global and local semantics from different perspectives. TRS-Net is a compound, high-resolution segmentation architecture equipped with learnable downsampler and upsampler modules to minimize information loss for optimal performance and efficiency. DmgFormer utilizes a transformer backbone and a convolutional decoder head with skip connections on a grid of crops aiming for high precision learning without downsizing. An augmented inference technique is used to boost performance further and reduce the possible loss of context due to grid cropping. Comprehensive experiments have been performed on the 3D physics-based graphics models (PBGMs) synthetic environments in the QuakeCity dataset. The proposed framework is evaluated using several metrics on three segmentation tasks: component type, component damage state, and global damage (crack, rebar, spalling). The models were developed as part of the 2^nd International Competition for Structural Health Monitoring.