• Journal of IKEEE
• /
• v.28 no.2
• /
• pp.209-220
• /
• 2024

A sketch is an intuitive means to express information, but compared to actual images, it has the problem of being highly abstract, diverse, and sparse. Recent advances in deep learning models have made it possible to discover features that are common to images and sketches. In this paper, we summarize recent trends in sketch-based image retrieval (SBIR) but it is not limited to SBIR. Besides SBIR, we also introduce sketch-based image recognition and generation studies. Zero-shot learning enables models to recognize categories not encountered during training. Zero-shot SBIR methods are also discussed. Commonly used free-hand sketch datasets are summarized and retrieval performance based on these datasets is reported.

• Journal of KIISE:Software and Applications
• /
• v.30 no.7_8
• /
• pp.703-712
• /
• 2003

With the explosive growth n the numbers and sizes of imaging technologies, Content-Based Image Retrieval (CBIR) has been attacked the interests of researchers in the fields of digital libraries, image processing, and database systems. In general, in the case of query-by-image, in user has to select an image from database to query, even though it is not his completely desired one. However, since query-by-sketch approach draws a query shape according to the user´s desire it can provide more high-level searching interface to the user compared to the query-b-image. As a result, query-by-sketch has been widely used. In this paper, we propose a Java-based image retrieval system that consists of sketch query and image classification. We use two features such as color histogram and Haar wavelets coefficients to search similar images. Then the Leave-One-Out method is used to classify database images. The categories of classification are photo & painting, city & nature, and sub-classification of nature image. By using the sketch query and image classification, w can offer convenient image retrieval interface to user and we can also reduce the searching time.

• The Journal of Korean Institute of Communications and Information Sciences
• /
• v.30 no.8C
• /
• pp.783-792
• /
• 2005

This paper proposes a feature extraction method for sketch-based image retrieval of animation character. We extract the specific regions using the detection of scene change and correlation points between two frames, and the property of animation production. We detect the area of focused similar colors in extracted specific region. And it is used as feature descriptor for image retrieval that focused color(FC) of regions, size, relation between FCs. Finally, an user can retrieve the similar character using property of animation production and user's sketch as a query Image.

• Journal of KIISE:Software and Applications
• /
• v.27 no.5
• /
• pp.551-562
• /
• 2000

Content-based image retrieval system retrieves an image from a database using visual features. Among approaches to express visual aspects in queries, 'query by sketch' is most convenient and expressive. However, every 'query by sketch' system has the query imperfectness problem. GContent-based image retrieval system retrieves an image from a database using visual features. Among approaches to express visual aspects in queries, 'query by sketch' is most convenient and expressive. However, every 'query by sketch' system has the query imperfectness problem. Generally, the query image produced by a user is different from the intended target image. To overcome this problem, many image retrieval systems use the spatial relationships of the objects, instead of pixel coordinates of the objects. In this paper, a query-converting algorithm for an image retrieval system, which uses the spatial relationship of every two objects as an image feature, is proposed. The proposed algorithm converts the query image into a graph that has the minimum number of edges, by eliminating every transitive edge. Since each edge in the graph represents the spatial relationship of two objects, the elimination of unnecessary edges makes the retrieval process more efficient. Experimental results show that the proposed algorithm leads the smaller number of comparison in searching process as compared with other algorithms that do not guarantee the minimum number of edges.

• Journal of Internet Computing and Services
• /
• v.10 no.6
• /
• pp.113-121
• /
• 2009

In order to retrieve images with different emotions in regions of the images, this paper proposes the image retrieval system using emotion sketch. The proposed retrieval system divides an image into $17{\times}17$ sub-regions and extracts emotion features in each sub-region. In order to extract the emotion features, this paper uses emotion colors on 160 emotion words from H. Nagumo's color scheme imaging chart. We calculate a histogram of each sub-region and consider one emotion word having the maximal value as a representative emotion word of the sub-region. The system demonstrates the effectiveness of the proposed emotion sketch and our experimental results show that the system successfully retrieves on the Corel image database.

• Journal of Internet Computing and Services
• /
• v.21 no.1
• /
• pp.33-43
• /
• 2020

Retrieving a 3D model from a 3D database and augmenting the retrieved model in the Augmented Reality system simultaneously became an issue in developing the plausible AR environments in a convenient fashion. It is considered that the sketch-based 3D object retrieval is an intuitive way for searching 3D objects based on human-drawn sketches as query. In this paper, we propose a novel deep learning based approach of retrieving a sketch-based 3D object as for an Augmented Reality Model. For this work, we introduce a new method which uses Sketch CNN, Wasserstein CNN and Wasserstein center loss for retrieving a sketch-based 3D object. Especially, Wasserstein center loss is used for learning the center of each object category and reducing the Wasserstein distance between center and features of the same category. The proposed 3D object retrieval and augmentation consist of three major steps as follows. Firstly, Wasserstein CNN extracts 2D images taken from various directions of 3D object using CNN, and extracts features of 3D data by computing the Wasserstein barycenters of features of each image. Secondly, the features of the sketch are extracted using a separate Sketch CNN. Finally, we adopt sketch-based object matching method to localize the natural marker of the images to register a 3D virtual object in AR system. Using the detected marker, the retrieved 3D virtual object is augmented in AR system automatically. By the experiments, we prove that the proposed method is efficiency for retrieving and augmenting objects.

• Journal of the Korean Institute of Telematics and Electronics B
• /
• v.33B no.7
• /
• pp.60-69
• /
• 1996

To retrieve complex data such as images in multimedia information, we need the content-based retrieval methods based on the visual properties rather than keywords. In this paper, a contrent-based image retrieval system is desinged and implemented to retrieve images using the features of images such as colors, lines and intensity vetor features when a visual query inputs. The contents for image retrievals are the color features extracted from the color component of 16 blocks of the image, th eline features extracted form 4 lines in the image and the shape features extracted from the intensity vectors of the 16 blocks. We can either use a whole image or a sketch image for query. As the experimental results demonstrate the precision 91% the recall 33% and the average rank 3.1 the retrieval performance is found to be high. The experimental results indicate that the retrieval using the weighted features have led to substantial improvement in the percision and performance of system.

• Journal of Information Management
• /
• v.42 no.3
• /
• pp.71-94
• /
• 2011

Query by visual example is the principal query paradigm for expressing queries in a content-based image retrieval environment. Query by image and query by sketch have long been purported as being viable methods of query formulation yet there is little empirical evidence to support their efficacy in facilitating query formulation. The ability of the searcher to express their information problem to an information retrieval system is fundamental to the retrieval process. The aim of this research was to investigate the query by image and query by sketch methods in supporting a range of information problems through a usability experiment in order to contribute to the gap in knowledge regarding the relationship between searchers' information problems and the query methods required to support efficient and effective visual query formulation. The results of the experiment suggest that query by image is a viable approach to visual query formulation. In contrast, the results strongly suggest that there is a significant mismatch between the searchers information problems and the expressive power of the query by sketch paradigm in supporting visual query formulation. The results of a usability experiment focusing on efficiency (time), effectiveness (errors) and user satisfaction show that there was a significant difference, p<0.001, between the two query methods on all three measures: time (Z=-3.597, p<0.001), errors (Z=-3.317, p<0.001), and satisfaction (Z=-10.223, p<0.001). The results also show that there was a significant difference in participants perceived usefulness of the query tools Z=-4.672, p<0.001.

• Journal of Internet Computing and Services
• /
• v.19 no.6
• /
• pp.91-99
• /
• 2018

Sketch-based 3D object retrieval is a convenient way to search for various 3D data using human-drawn sketches as query. In this paper, we propose a new method of using Sketch CNN, Wasserstein CNN and Wasserstein center loss for sketch-based 3D object search. Specifically, Wasserstein center loss is a method of learning the center of each object category and reducing the Wasserstein distance between center and features of the same category. To do this, the proposed 3D object retrieval is performed as follows. Firstly, Wasserstein CNN extracts 2D images taken from various directions of 3D object using CNN, and extracts features of 3D data by computing the Wasserstein barycenters of features of each image. Secondly, the features of the sketch are extracted using a separate Sketch CNN. Finally, we learn the features of the extracted 3D object and the features of the sketch using the proposed Wasserstein center loss. In order to demonstrate the superiority of the proposed method, we evaluated two sets of benchmark data sets, SHREC 13 and SHREC 14, and the proposed method shows better performance in all conventional metrics compared to the state of the art methods.

• Journal of Korea Society of Industrial Information Systems
• /
• v.9 no.4
• /
• pp.26-31
• /
• 2004

Due to the recent explosive progress of Web, We can easily access a large number of images from m. In this paper, we describe our approach of developing a general purpose content based image retrieval system over the H using a Web agent. The Web agent extracts text information of images from the links and file contents in HTML. The proposed system retrieves the images from database using the query by sketch and the query by example on Web browser. Experimental results demonstrate the effectiveness of the new approach.