Browse > Article
http://dx.doi.org/10.3837/tiis.2019.08.028

Stochastic Non-linear Hashing for Near-Duplicate Video Retrieval using Deep Feature applicable to Large-scale Datasets  

Byun, Sung-Woo (Department of Computer Science, Graduate School, SangMyung University)
Lee, Seok-Pil (Department of Computer Science, Graduate School, SangMyung University)
Publication Information
KSII Transactions on Internet and Information Systems (TIIS) / v.13, no.8, 2019 , pp. 4300-4314 More about this Journal
Abstract
With the development of video-related applications, media content has increased dramatically through applications. There is a substantial amount of near-duplicate videos (NDVs) among Internet videos, thus NDVR is important for eliminating near-duplicates from web video searches. This paper proposes a novel NDVR system that supports large-scale retrieval and contributes to the efficient and accurate retrieval performance. For this, we extracted keyframes from each video at regular intervals and then extracted both commonly used features (LBP and HSV) and new image features from each keyframe. A recent study introduced a new image feature that can provide more robust information than existing features even if there are geometric changes to and complex editing of images. We convert a vector set that consists of the extracted features to binary code through a set of hash functions so that the similarity comparison can be more efficient as similar videos are more likely to map into the same buckets. Lastly, we calculate similarity to search for NDVs; we examine the effectiveness of the NDVR system and compare this against previous NDVR systems using the public video collections CC_WEB_VIDEO. The proposed NDVR system's performance is very promising compared to previous NDVR systems.
Keywords
Near-duplicate Video; NDVR; Class Activation Maps; Hashing; supervised learning;
Citations & Related Records
연도 인용수 순위
  • Reference
1 Liu, J., Huang, Z., Cai, H., Shen, H. T., Ngo, C. W., and Wang, W., "Near-duplicate video retrieval: Current research and future trends," ACM Comput. Surv., vol. 45, no. 4, Art. no. 44., pp. 218-227, 2013.
2 J. Song, Y. Yang, Z. Huang, H. T. Shen, and R. Hong, "Multiple feature hashing for real-time large scale near-duplicate video retrieval," in Proc. of 19th ACM Int. Conf. Multimedia, pp. 423-432, 2011
3 M. Cherubini, R. De Oliveira, and N. Oliver, "Understanding near-duplicate videos: A user-centric approach," in Proc. of 17th ACM Int. Conf. Multimedia, pp. 35-44, 2009.
4 H. T. Shen, X. Zhou, Z. Huang, J. Shao, and X. Zhou, "UQLIPS: A real-time near-duplicate video clip detection system," in Proc. 33rd Int. Conf. Very Large Data Bases, pp. 1374-1377, 2007.
5 H.-K. Tan, C.-W. Ngo, R. Hong, and T.-S. Chua, "Scalable detection of partial near-duplicate videos by visual-temporal consistency," in Proc. of 17th ACM Int. Conf. Multimedia, pp. 145-154, 2009.
6 X. Wu, A. G. Hauptmann, and C.-W. Ngo, "Practical elimination of near-duplicates from web video search," in Proc. of 15th ACM Int. Conf. Multimedia, pp. 218-227, 2007.
7 Yanbin Hao, Tingting Mu, Richang Hong, Meng Wang, Ning An, John Y. Goulermas, “Stochastic Multiview Hashing for Large-Scale Near-Duplicate Video Retrieval,” IEEE Transactions on Multimedia, Vol. 19, No. 1, pp. 1-14, 2016.   DOI
8 L. Shang, L. Yang, F. Wang, K.-P. Chan, and X.-S. Hua, "Real-time large scale near-duplicate web video retrieval," in Proc. of 18th ACM Int. Conf. Multimedia, pp. 531-540, 2010.
9 D. Zhang, J. Wang, D. Cai, and J. Lu, "Self-taught hashing for fast similarity search," in Proc. of 33rd Int. ACM SIGIR Conf. Res. Develop. Inf. Retrieval, pp. 18-25, 2010.
10 Y. Weiss, A. Torralba, and R. Fergus, "Spectral hashing," in Proc. of Adv. Neural Inf. Process. Syst. Conf., pp. 1753-1760, 2009.
11 J. Yuan, L.-Y. Duan, Q. Tian, S. Ranganath, and C. Xu, "Fast and robust short video clip search for copy detection," in Proc. of Adv. Multimedia Inf. Process. Conf., pp. 479-488, 2004.
12 G. Zhao and M. Pietikainen, "Dynamic texture recognition using local binary patterns with an application to facial expressions," IEEE Trans. Pattern Anal. Mach. Intell., vol. 29, no. 6, pp. 915-928, Jun. 2007.   DOI
13 D.-G. Lowe, "Distinctive image features from scale-invariant keypoints," Int. J. Comput. Vis., vol. 60, no. 2, pp. 91-110, 2004.   DOI
14 D.-G. Lowe, "Object recognition from local scale-invariant features," in Proc. of Int. Conf. Comput. Vis., pp. 1150-1157, 1999.
15 Y. Ke and R. Sukthankar, "PCA-SIFT: A more distinctive representation for local image descriptors," in Proc. of IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recog., pp. 506-513, Jun.-Jul. 2004.
16 Chenggang Yan, Liang Li, Chunjie Zhang, Bingtao Liu, Yongdong Zhang, Qionghai Dai, "Cross-modality Bridging and Knowledge Transferring for Image Understanding," IEEE Transactions on Multimedia. (Early Access), pp. 1-1, 2019
17 Chenggang Yan, Liang Li, Chunjie Zhang, Bingtao Liu, Yongdong Zhang, Qionghai Dai, "A Fast Uyghur Text Detector for Complex Background Images," IEEE Transactions on Multimedia,Vol. 20, Issue. 12, pp. 3389-3398, 2018.   DOI
18 W. Liu, J.Wang, R. Ji, Y.-G. Jiang, and S.-F. Chang, "Supervised hashing with kernels," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., pp. 2074-2081, 2012.
19 J. Song, L. Gao, Y. Yan, D. Zhang, and N. Sebe, "Supervised hashing with pseudo labels for scalable multimedia retrieval," in Proc. of 23rd ACM Int. Conf. Multimedia, pp. 827-830, 2015.
20 R. Salakhutdinov and G. E. Hinton, "Learning a nonlinear embedding by preserving class neighbourhood structure," in Proc. of 11th Int. Conf. Artif. Intell. Statist., pp. 412-419, 2007.
21 A. Gionis et al., "Similarity search in high dimensions via hashing," in Proc. of 25th Int. Conf. Very Large Data Bases, pp. 518-529, 1999.
22 J. Song, Y. Yang, Z. Huang, H. T. Shen, and J. Luo, "Effective multiple feature hashing for large-scale near-duplicate video retrieval," IEEE Trans. Multimedia, vol. 15, no. 8, pp. 1997-2008, Dec. 2013.   DOI
23 David G. Lowe, "Distinctive Image Features from Scale-Invariant Keypoints," International Journal of Computer Vision, vol. 60, no. 2, pp. 91-110, Nov. 2004.   DOI
24 P. Viola, M. Jones, "Rapid object detection using a boosted cascade of simple features," in Proc. of IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001, 2001.
25 Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, Antonio Torralba, "Learning Deep Features for Discriminative Localization," in Proc. of Computer Vision and Pattern Recognition (CVPR), 2016 IEEE Conference on, PP. 2921-2929, 2016.
26 Mustafa Ozuysal, Michael Calonder, Vincent Lepetit, Pascal Fua, "Fast Keypoint Recognition Using Random Ferns," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 32, no. 3, pp. 448-461, Jan. 2009.   DOI