Browse > Article
http://dx.doi.org/10.15701/kcgs.2014.20.3.1

Composition of Foreground and Background Images using Optical Flow and Weighted Border Blending  

Gebreyohannes, Dawit (Ajou University)
Choi, Jung-Ju (Ajou University)
Publication Information
Journal of the Korea Computer Graphics Society / v.20, no.3, 2014 , pp. 1-8 More about this Journal
Abstract
We propose a method to compose a foreground object into a background image, where the foreground object is a part (or a region) of an image taken by a front-facing camera and the background image is a whole image taken by a back-facing camera in a smart phone at the same time. Recent high-end cell-phones have two cameras and provide users with preview video before taking photos. We extract the foreground object that is moving along with the front-facing camera using the optical flow during the preview. We compose the extracted foreground object into a background image using a simple image composition technique. For better-looking result in the composed image, we apply a border smoothing technique using a weighted-border mask to blend transparency from background to foreground. Since constructing and grouping pixel-level dense optical flow are quite slow even in high-end cell-phones, we compute a mask to extract the foreground object in low-resolution image, which reduces the computational cost greatly. Experimental result shows the effectiveness of our extraction and composition techniques, with much less computational time in extracting the foreground object and better composition quality compared with Poisson image editing technique which is widely used in image composition. The proposed method can improve limitedly the color bleeding artifacts observed in Poisson image editing using weighted-border blending.
Keywords
Optical flow; Image composition; Foreground extraction; Smartphone camera;
Citations & Related Records
연도 인용수 순위
  • Reference
1 M. Piccardi, "Background subtraction techniques: a review," in IEEE International Conference on Systems, Man and Cybernetics, vol. 4, Oct 2004, pp. 3099-3104 vol.4.
2 I. Kartika and S. Mohamed, "Frame differencing with postprocessing techniques for moving object detection in outdoor environment," in IEEE 7th International Colloquium on Signal Processing and its Applications (CSPA), March 2011, pp. 172-176.
3 J.Weber and J. Malik, "Robust computation of optical flow in a multi-scale differential framework," in Fourth International Conference on Computer Vision, May 1993, pp. 12-20.
4 M. Grundland, R. Vohra, G. P. Williams, and N. A. Dodgson, "Nonlinear multiresolution image blending," International Journal of Machine Graphics & Vision, vol. 15, no. 3, pp. 381-390, Jan. 2006. [Online]. Available: http://dl.acm.org/citation.cfm?id=1375858.1375874
5 P. P´erez, M. Gangnet, and A. Blake, "Poisson image editing," ACM Trans. Graph., vol. 22, no. 3, pp. 313-318, July 2003. [Online]. Available: http://doi.acm.org/10.1145/882262.882269   DOI   ScienceOn
6 J. Jia, J. Sun, C.-K. Tang, and H.-Y. Shum, "Dragand- drop pasting," ACM Trans. Graph., vol. 25, no. 3, pp. 631-637, July 2006. [Online]. Available: http://doi.acm.org/10.1145/1141911.1141934   DOI   ScienceOn
7 M. Kazhdan and H. Hoppe, "Streaming multigrid for gradient-domain operations on large images," ACM Trans. Graph., vol. 27, no. 3, pp. 21:1-21:10, Aug. 2008. [Online]. Available: http://doi.acm.org/10.1145/1360612.1360620
8 A. Agarwala, "Efficient gradient-domain compositing using quadtrees," ACM Trans. Graph., vol. 26, no. 3, July 2007. [Online]. Available: http://doi.acm.org/10.1145/1276377.1276495
9 Z. Farbman, G. Hoffer, Y. Lipman, D. Cohen- Or, and D. Lischinski, "Coordinates for instant image cloning," ACM Trans. Graph., vol. 28, no. 3, pp. 67:1-67:9, July 2009. [Online]. Available: http://doi.acm.org/10.1145/1531326.1531373
10 S. Jeschke, D. Cline, and P.Wonka, "A GPU Laplacian solver for diffusion curves and Poisson image editing," ACM Trans. Graph., vol. 28, no. 5, pp. 116:1-116:8, Dec. 2009. [Online]. Available: http://doi.acm.org/10.1145/1618452.1618462
11 J. Sun, J. Jia, C.-K. Tang, and H.-Y. Shum, "Poisson matting," ACM Trans. Graph., vol. 23, no. 3, pp. 315-321, Aug. 2004. [Online]. Available: http://doi.acm.org/10.1145/1015706.1015721   DOI   ScienceOn
12 T. Georgiev, "Photoshop healing brush: a tool for seamless cloning." [Online]. Available: http://www.tgeorgiev.net/Photoshop Healing.pdf
13 Samsumg galaxy s4: Dual shot. [Online]. Available: http://www.samsung.com/global/microsite/galaxys4/fun.html #page=dualshot
14 C. Zach, T. Pock, and H. Bischof, "A duality based approach for realtime TV-L1 optical flow," in Proceedings of the 29th DAGM Conference on Pattern Recognition. Berlin, Heidelberg: Springer-Verlag, 2007, pp. 214-223. [Online]. Available: http://dl.acm.org/citation.cfm?id=1771530.1771554
15 Y. Tanaka, M. Hasegawa, and S. Kato, "Improved image concentration for artifact-free image dilution and its application to image coding," in 17th IEEE International Conference on Image Processing (ICIP), Sept 2010, pp. 1225-1228.