Browse > Article
http://dx.doi.org/10.1633/JIM.2011.42.3.071

Query by Visual Example: A Comparative Study of the Efficacy of Image Query Paradigms in Supporting Visual Information Retrieval  

Venters, Colin C. (Dept. of Library and Information Science, Kyungpook National University)
Publication Information
Journal of Information Management / v.42, no.3, 2011 , pp. 71-94 More about this Journal
Abstract
Query by visual example is the principal query paradigm for expressing queries in a content-based image retrieval environment. Query by image and query by sketch have long been purported as being viable methods of query formulation yet there is little empirical evidence to support their efficacy in facilitating query formulation. The ability of the searcher to express their information problem to an information retrieval system is fundamental to the retrieval process. The aim of this research was to investigate the query by image and query by sketch methods in supporting a range of information problems through a usability experiment in order to contribute to the gap in knowledge regarding the relationship between searchers' information problems and the query methods required to support efficient and effective visual query formulation. The results of the experiment suggest that query by image is a viable approach to visual query formulation. In contrast, the results strongly suggest that there is a significant mismatch between the searchers information problems and the expressive power of the query by sketch paradigm in supporting visual query formulation. The results of a usability experiment focusing on efficiency (time), effectiveness (errors) and user satisfaction show that there was a significant difference, p<0.001, between the two query methods on all three measures: time (Z=-3.597, p<0.001), errors (Z=-3.317, p<0.001), and satisfaction (Z=-10.223, p<0.001). The results also show that there was a significant difference in participants perceived usefulness of the query tools Z=-4.672, p<0.001.
Keywords
Content-based Image Retrieval; Image Retrieval; Query by Image; Query by Visual Example; Query by Sketch; Visual Information Retrieval; Usability;
Citations & Related Records
연도 인용수 순위
  • Reference
1 ISO 9241. 1998. International Organization for Standardization.
2 Jaimes, A. and S-F. Chang. 2002. "Concepts and Techniques for Indexing Visual Concepts." In: Castelli, V. and Bergman, L. D. (ed). Image Databases: Search and Retrieval of Digital Imagery, John Wiley & Sons Inc. : 497.
3 Jaimes, A., N. Sebe, and D. Gatica-Perez. 2006. "Human-centered Computing: A Multimedia Perspective." Proceedings of the 14th annual ACM international conference on Multimedia. Santa Barbara, CA, USA, October 23-27, 2006, ACM: 855-864.
4 Kato, T. 1991. "Intelligent Visual Interaction with Image Database Systems: Toward the Multimedia Personal Interface." Journal of Information Processing, 14(2): 134-143.
5 Kato, T., T. Kurita, and H. Shimogaki. 1989. "Multimedia Interaction with Image Database Systems." In: Advanced Database System Symposium '89, Kyoto, Japan, 1989: 271-278.
6 Kimia, B. B. 2002. "Shape Representation for Image Retrieval." In: Castelli, V. and Bergman, L. D. (eds). Image Databases: Search and Retrieval of Digital Imagery, John Wiley & Sons Inc. : 345-372.
7 Ko, B. C. and H. Byun. 2002. "Query-by- Gesture: An Alternative Content-Based Image Retrieval Query Scheme." Journal of Visual Languages and Computing, 13(4): 375-390.   DOI   ScienceOn
8 Ko, B. C., J. Peng, and H. Byun. 2001. "Region- Based Image Retrieval Using Probabilistic Feature Relevance Learning." Pattern Analysis and Applications, 4(2-3): 174- 184.   DOI
9 Laaksonen, J., M. Koskela, S. Laakso, and E. Oja. 2000. "PicSOM: Content-based Image Retrieval with Self-organizing maps." Pattern Recognition Letters, 21(3- 14): 1199-1207.   DOI   ScienceOn
10 Korfhage, R. R. 1997. Information Storage and Retrieval. New York: John Wiley.
11 Lai, T-S. 2000. "CHROMA: A Photographic Image Retrieval System." Ph.D. Thesis. University of Sunderland, UK. January 2000.
12 Blaser, A. D. 1997. "User Interaction in a Sketch-Based GIS User Interface." Lecture Notes In Computer Science, 1329 (1997): 505-506.
13 Lew, M. S. and T. S. Huang. 2001. "Visual Information Retrieval: Paradigms, Applications, and Research Issues." In: Lew, M. S. (ed.) Principles of Visual Information Retrieval. Springer-Verlag, Advances in Pattern Recognition: 3-9.
14 Loncaric, S. 1998. "A Survey of Shape Analysis Techniques." Pattern Recognition, 31(8): 983-1001.   DOI   ScienceOn
15 Preece, J., Y. Rogers, H. Sharp, D. Benyon, S. Holland, and T. Carey. 1994. Humancomputer Interaction. Boston: Pearson Addison Wesley.
16 Mingqiang, Y., K. Kidiyo, and R. Joseph. 2008. A Survey of Shape Feature Extraction Techniques. Pattern Recognition Techniques, Technology and Applications, P-Y. Yin (Ed.), InTech: 43-90.
17 Muller, W., H. Müller, S. Marchand-Maillet, T. Pun, D. M. Squire, Z. Pecenovic, C. Giess, and A. P. de Vries. 2000. "MRML: An Extensible Communication Protocol for Interoperability and Bench-marking of Multimedia Information Retrieval Systems." Visual Information and Information Systems: 300-311.
18 Berlin, B. and P. Kay. 1991. Basic Color Terms: Their Universality and Evolution. Berkeley: University of California Press.
19 Nakazato M. and T. S. Huang. 2001. "An Interactive 3d Visualization for Content-based Image Retrieval." Proceedings of the International Conference on Multimedia and Expo, Waseda University, Tokyo, Japan, 22-25 August 2001: 44-47.
20 Bird, C., P. J. Elloitt, and P. M. Hayward. 1999. "Content-Based Retrieval for European Image Libraries." In: The Challenge of Image Retrieval: CIR99, The 2nd UK Conference on Image Retrieval, Forte Posthouse Hotel, Newcastle upon Tyne, United Kingdom, 25th-26th February 1999, edited by D. J. Harper and J. P. Eakins.
21 Bordogna, G. and M. Pagani. 2010. "A Flexible Content-based Image Retrieval Model and Customizable System for the Retrieval of Shapes." Journal of the American Society for Information Science and Technology, 61(5): 907-926.   DOI   ScienceOn
22 Boujemaa, N., J. Fauqueur, and V. Gouet. 2004. "What's Beyond Query by Example?" Lecture Notes in Computer Science, Springer-Verlag.
23 Yoshitaka, A. and T. Ichikawa. 1999. "A Survey on Content-Based Retrieval for Multimedia Databases." IEEE Transactions on Knowledge and Data Engineering, 11(1): 81-93.   DOI   ScienceOn
24 Rui, Y. and T. S. Huang. 2001. "Relevance Feedback Techniques." In: Lew, M. S. (ed.) Principles of Visual Information Retrieval. Springer-Verlag, Advances in Pattern Recognition: 219-258.
25 Santini, S. and R. Jain. 1997. "Image Databases are not Databases with Images." Proceedings of the 9th International Conference on Image Analysis and Processing. Volume II: 38-45.
26 Smeulders, A. W. M., M. Worring, S. Santini, A. Gupta, and R. Jain. 2000. "Contentbased Image Retrieval at the End of the Early Years." IEEE Transaction on Pattern Analysis and Machine Intelligence, 22(12): 1349-1380.   DOI   ScienceOn
27 Castelli, V. and L. D. Bergman. 2002. Image Databases: Search and Retrieval of Digital Imagery. New York: John Wiley, 1-10.
28 Veltkamp, R. C. and M. Hagedoorn. 2001. "State of the Art in Shape Matching." In: Lew, M. S. (ed.) Principles of Visual Information Retrieval. Springer-Verlag, Advances in Pattern Recognition: 87-120.
29 Vendrig, J., M. Worring, and A. W. M. Smeulders. 2001. "Filter Image Browsing: Interactive Image Retrieval by Using Database Overviews." Multimedia Tools and Applications, 15(1): 83-103.   DOI   ScienceOn
30 Wilson, T. D. 1981. "On User Studies and Information Needs." Journal of Documentation, 37: 3-15.   DOI   ScienceOn
31 Eakins, J. P., J. M. Boardman, and M. E. Graham. 1998. "Similarity Retrieval of Trademark Images." IEEE MultiMedia, 5(2): 53-63.   DOI   ScienceOn
32 Chin, J., V. Diehl, and L. Norman. 1988. "Development of an Instrument Measuring User Satisfaction of the Human- Computer Interface." In: CHI '88: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Washington, D.C. USA, 15-19 May 1988, edited by J. J. O'Hare, ACM Press: 213-218.
33 Datta, R., D. Joshi, J. Li, and J. Z. Wang. 2008. "Image Retrieval: Ideas, Influences, and Trends of the New Age." ACM Computing Surveys, 40(2).
34 Del Bimbo, A. and P. Pala. 1997. "Visual Image Retrieval by Elastic Matching of User Sketches." IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(2): 121-132.   DOI   ScienceOn
35 Di Lecce, V and A. Guerriero. 1999. "An Evaluation of the Effectiveness of Image Features for Image Retrieval." Journal of Visual Communication and Image Representation, 10(4): 351-362.   DOI   ScienceOn
36 Eakins, J. P. 1992. "Pictorial Information Systems: Prospects and Problems." Proceedings of 14th BCS Information Retrieval Specialist Group Research Colloquium, Lancaster, April 1992: 102-123.
37 Eidenberger, H. and C. Breiteneder. 2003. "VizIR: A Framework for Visual Information Retrieval." Journal of Visual Languages and Computing, 14(5): 443- 469.   DOI   ScienceOn
38 Enser, P. G. B. 1995. "Progress in Documentation Pictorial Information Retrieval." Journal of Documentation, 51(2): 126- 170.   DOI   ScienceOn
39 Flickner, M., H. Sawhney, W. Niblack, J. Ashley, Q. Huang, B. Dom, M. Gorkani, J. Hafner, D. Lee, D. Petkovic, D. Steele, and P. Yanker. 1995. "Query by Image and Video Content: The QBIC System." IEEE Computer, 28(9): 23-32.   DOI   ScienceOn
40 Feng, D., W. C. Siu, and H. J. Zhang. 2003. Multimedia Information Retrieval and Management: Technological Fundamentals and Applications. New York: Springer, 1-26.
41 Idris, F. and S. Panchanathan. 1997. "Review of Image and Video Indexing Techniques." Journal of Visual Communication and Image Representation, 8(2): 146-166.   DOI   ScienceOn
42 Frokjaer, E., M. Hertzum, and K. Hornbaek. 2000. "Measuring Usability: Are Effectiveness, Efficiency, and Satisfaction Really Correlated?" Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, The Hague, The Netherlands, 01- 06, April, 2000, ACM Press: 345-352.
43 Gecsei, J. and D. Martin. 1989. "Browsing Assess to Visual Information." Optical Information Systems, 9(5): 237-241.
44 Gudivada, V. N. and V. V. Raghaven. 1995. "Content-based Image Retrieval Systems." IEEE Computer, September 1995: 20.
45 Hanjalic, A., N. Sebe, and E. Y. Chang. 2006. "Multimedia Content Analysis, Management and Retrieval: Trends and Challenges." In: Electronic Imaging 2006, Multimedia Content Analysis, Management and Retrieval 2006, IS&T/SPIE International Symposium, SPIE Vol. 6073, San Jose Marriott and San Jose Convention Centre, San Jose, California, USA, 15-19 January 2006, edited by Edward Y. Chang, Alan Hanjalic and Nicu Sebe, SPIE.
46 Hirata, K. and T. Kato. 1992. "Query by Visual Example." Proceedings of the 3rd International Conference on Extending Database Technology, Vienna, Austria, 23rd-27th March 1992, edited by A. Pirotte, C. Delobel and G. Gottlob, Springer-Verlag: 56-71.
47 Hix, D. and H. R. Hartson. 1993. Developing User Interfaces: Ensuring Usability Through Product and Process. New York: John Wiley.