• Journal of International Society for Simulation Surgery
• /
• v.1 no.2
• /
• pp.57-61
• /
• 2014

We present a robust 3D facial reconstruction method using a single image generated by face-specific super resolution technique. Based on the several consecutive frames with low resolution, we generate a single high resolution image and a three dimensional facial model based on it. To do this, we apply PME method to compute patch similarities for SR after two-phase warping according to facial attributes. Based on the SRI, we extract facial features automatically and reconstruct 3D facial model with basis which selected adaptively according to facial statistical data less than a few seconds. Thereby, we can provide the facial image of various points of view which cannot be given by a single point of view of a camera.

• Journal of KIISE
• /
• v.43 no.9
• /
• pp.1034-1042
• /
• 2016

Together with machine-learning frameworks, area-based imagery registration techniques can be easily applied to diverse types of image pairs without predefined features and feature descriptors. However, feature detectors are often used to quickly identify candidate image patch pairs, limiting the applicability of these registration techniques. In this paper, we propose a ConvNet (Convolutional Network) "Dart" that provides not only the matching metric between patches, but also information about their distance, which are helpful in reducing the search space of the corresponding patch pairs. In addition, we propose a ConvNet "Fad" to identify the patches that are difficult for Dart to improve the accuracy of registration. These two networks were successfully implemented using Deep Learning with the help of a number of training instances generated from a few registered image pairs, and were successfully applied to solve a simple image registration problem, suggesting that this line of research is promising.

• Journal of the Institute of Electronics Engineers of Korea CI
• /
• v.42 no.1
• /
• pp.67-78
• /
• 2005

We proposed a blind watermarking for 3D mesh model using the patch CEGIs. The CEGI is the 3D orientation histogram with complex weight whose magnitude is the mesh area and phase is the normal distance of the mesh from the designated origin. In the proposed algorithm we divide the 3D mesh model into the number of patch that determined adaptively to the shape of model and calculate the patch CEGIs. Some cells for embedding the watermark are selected according to the rank of their magnitudes in each of patches after calculating the respective magnitude distributions of CEGI for each patches of a mesh model. Each of the watermark bit is embedded into cells with the same rank in these patch CEGI. Based on the patch center point and the rank table as watermark key, watermark extraction and realignment process are performed without the original mesh. In the rotated model, we perform the realignment process using Euler angle before the watermark extracting. The results of experiment verify that the proposed algorithm is imperceptible and robust against geometrical attacks of cropping, affine transformation and vertex randomization as well as topological attacks of remeshing and mesh simplification.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.11 no.4
• /
• pp.2109-2123
• /
• 2017

This paper presents a super-resolution reconstruction method based on sparse representation for two-dimensional bar code images. Considering the features of two-dimensional bar code images, Kirsch and LBP (local binary pattern) operators are used to extract the edge gradient and texture features. Feature extraction is constituted based on these two features and additional two second-order derivatives. By joint dictionary learning of the low-resolution and high-resolution image patch pairs, the sparse representation of corresponding patches is the same. In addition, the global constraint is exerted on the initial estimation of high-resolution image which makes the reconstructed result closer to the real one. The experimental results demonstrate the effectiveness of the proposed algorithm for two-dimensional bar code images by comparing with other reconstruction algorithms.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.13 no.8
• /
• pp.3962-3980
• /
• 2019

To deal with single sample face recognition, this paper presents a patch based semi-supervised linear regression (PSLR) algorithm, which draws facial variation information from unlabeled samples. Each facial image is divided into overlapped patches, and a regression model with mapping matrix will be constructed on each patch. Then, we adjust these matrices by mapping unlabeled patches to $[1,1,{\cdots},1]^T$. The solutions of all the mapping matrices are integrated into an overall objective function, which uses ${\ell}_{2,1}$-norm minimization constraints to improve discrimination ability of mapping matrices and reduce the impact of noise. After mapping matrices are computed, we adopt majority-voting strategy to classify the probe samples. To further learn the discrimination information between probe samples and obtain more robust mapping matrices, we also propose a multistage PSLR (MPSLR) algorithm, which iteratively updates the training dataset by adding those reliably labeled probe samples into it. The effectiveness of our approaches is evaluated using three public facial databases. Experimental results prove that our approaches are robust to illumination, expression and occlusion.

• Journal of the Institute of Electronics Engineers of Korea SP
• /
• v.48 no.6
• /
• pp.48-60
• /
• 2011

Nowadays, the 3D community is actively researching on 3D imaging and free-viewpoint video (FVV). The free-viewpoint rendering in multi-view video, virtually move through the scenes in order to create different viewpoints, has become a popular topic in 3D research that can lead to various applications. However, there are restrictions of cost-effectiveness and occupying large bandwidth in video transmission. An alternative to solve this problem is to generate virtual views using a single texture image and a corresponding depth image. A critical issue on generating virtual views is that the regions occluded by the foreground (FG) objects in the original views may become visible in the synthesized views. Filling this disocclusions (holes) in a visually plausible manner determines the quality of synthesis results. In this paper, a new approach for handling disocclusions using depth based inpainting algorithm in synthesized views is presented. Patch based non-parametric texture synthesis which shows excellent performance has two critical elements: determining where to fill first and determining what patch to be copied. In this work, a noise-robust filling priority using the structure tensor of Hessian matrix is proposed. Moreover, a patch matching algorithm excluding foreground region using depth map and considering epipolar line is proposed. Superiority of the proposed method over the existing methods is proved by comparing the experimental results.

• Journal of the HCI Society of Korea
• /
• v.2 no.2
• /
• pp.11-18
• /
• 2007

We propose a novel method to conceal fiducial markers observed in augmented scenes using patch-based texture synthesis. Despite the efficiency for simple object recognition and tracking, the markers deliver inherent obtrusiveness. They do not only reduce immersiveness, but also severely degrade usability of augmented reality. The proposed method constructs alternative images in real time to overlay markers present in the sequence of images. The global characteristics of background textures are retained and the results are more adaptive to illumination changes.

• Proceedings of the IEEK Conference
• /
• 1998.06a
• /
• pp.773-776
• /
• 1998

In this paper, we propose an efficient multi-view images coding algorithm which finds the optimal texture from the restricted number of multi-view images. The X-Y plane of the normalized object space is divided into triangular patches. The depth value of the node is determined by applying the block based disparity compensation method and then the texture of the each patch is extracted by applying the affine transformation patch is extracted by applying the affine transformation based disparity compensation method to the multi-view images. We restricted the number of images contributed to determining the texture comapred to traditional methods which use all the multi-view images in the texture extraction. Experimental results show that the SNR of images encoded by the proposed algorithm is better than that of imaes encoded by the traditional method by the amount about 0.2dB for the test sets of multi-view images called dragon, kid, city and santa. The recovered images from the encoded data by the proposed method show the better visual images than the recovered images from the encoded data by the traditional methods.

• Journal of Information Processing Systems
• /
• v.15 no.6
• /
• pp.1449-1461
• /
• 2019

Face sketch synthesis plays an important role in public security and digital entertainment. In this paper, we present a novel face sketch synthesis method via local similarity and nonlocal similarity regularization terms. The local similarity can overcome the technological bottlenecks of the patch representation scheme in traditional learning-based methods. It improves the quality of synthesized sketches by penalizing the dissimilar training patches (thus have very small weights or are discarded). In addition, taking the redundancy of image patches into account, a global nonlocal similarity regularization is employed to restrain the generation of the noise and maintain primitive facial features during the synthesized process. More robust synthesized results can be obtained. Extensive experiments on the public databases validate the generality, effectiveness, and robustness of the proposed algorithm.

• Smart Structures and Systems
• /
• v.29 no.1
• /
• pp.29-39
• /
• 2022

Crack detection plays an important role in the maintenance and protection of steel box girder of bridges. However, since the cracks only occupy an extremely small region of the high-resolution images captured from actual conditions, the existing methods cannot deal with this kind of image effectively. To solve this problem, this paper proposed a novel three-stage method based on deep learning technology and morphology operations. The training set and test set used in this paper are composed of 360 images (4928 × 3264 pixels) in steel girder box. The first stage of the proposed model converted high-resolution images into sub-images by using patch-based method and located the region of cracks by CBAM ResNet-50 model. The Recall reaches 0.95 on the test set. The second stage of our method uses the Attention U-Net model to get the accurate geometric edges of cracks based on results in the first stage. The IoU of the segmentation model implemented in this stage attains 0.48. In the third stage of the model, we remove the wrong-predicted isolated points in the predicted results through dilate operation and outlier elimination algorithm. The IoU of test set ascends to 0.70 after this stage. Ablation experiments are conducted to optimize the parameters and further promote the accuracy of the proposed method. The result shows that: (1) the best patch size of sub-images is 1024 × 1024. (2) the CBAM ResNet-50 and the Attention U-Net achieved the best results in the first and the second stage, respectively. (3) Pre-training the model of the first two stages can improve the IoU by 2.9%. In general, our method is of great significance for crack detection.