Video Content Manipulation Using 3D Analysis for MPEG-4

This paper is concerned with realistic mainpulation of content in video sequences. Manipulation of content in video sequences is one of the content-based functionalities for MPEG-4 Visual standard. We present an approach to synthesizing video sequences by using the intermediate outputs of three-dimensional (3D) motion and depth analysis. For concreteness, we focus on video showing 3D motion of an observer relative to a scene containing planar runways (or roads). We first present a simple runway (or road) model. Then, we describe a method of identifying the runway (or road) boundary in the image using the Point of Heading Direction (PHD) which is defined as the image of, the ray along which a camera moves. The 3D motion of the camera is obtained from one of the existing 3D analysis methods. Then, a video sequence containing a runway is manipulated by (i) coloring the scene part above a vanishing line, say blue, to show sky, (ii) filling in the occluded scene parts, and (iii) overlaying the identified runway edges and placing yellow disks in them, simulating lights. Experimental results for a real video sequence are presented.