Although digitalization of broadcast waves is extending the multichannel programming capabilities of broadcasting, it does not fully exploit the potentiality of the digital evolution in broadcasting. ISDB has been put forth as second-generation digital broadcasting which offers a basis for a universal service in the future multimedia environment, and opens many new possibilities for services and their use. The paper is divided into three parts: A vision of the future of broadcasting, Evolution towards ISDB - integrated Services Television, and Technology development.

Each reproducing speech signal has each particular signal property, because of the processing of encoding and decoding for communications through various media. In this paper, we examine the correlation between speech signal quality and sensory pleasure for the sensory improvement of that signal. In experiments, we evaluate the quality of speech signals through various media by psychological auditory test and physical features of these signals.

This paper describes an image segmentation technique for the object-oriented coding at very low bit rates. By noting that, in the object-oriented coding technique, each objects are represented by 3 parameters, namely, shape, motion, and color informations, we propose a segmentation technique, in which the 3 parameters are fully exploited. To achieve this goal, starting with the color space conversion and the noise reduction, the input image is divided into many small regions by the K-menas algorithm on the O-K-S color space. Then, each regions are merged, according to the shape and motion information. In simultations, it is shown that the proposed technique segments the input image into relevant objects, according to the shape and motion as well as the colors. In addition, in order to evaluate the performance of the proposed technique, we introduce the notion of the interesting regions, and provide the results of encoding the image with emphasizing the interesting regions.

The present block-based DCT encoder transforms images regardless of layers and then simply partitions the transformed data into a few layers, for example low and high frequency bands in JPEG. Yet, it fails to utilize the similarity of coefficients in each band. Therefore, we combine the subband coder and the block-based DCt coder in this paper. The new coding scheme enables the data to automatically be classified into several layers and increases the efficiency of transform. Various possible coding structures are investigated and the simulation results are also provided.

Bit allocation is a critical problem in video encoding such as MPEG. To improve the quality of the reconstructed sequence for a given bit rate, the assigned target bits for a group of pictures (GOP) must be allocated to each picture efficiently. In this paper, we derive a target bit allocation algorithm for more efficient rate control, by assuming that the average rate-distortion curve for an input source is logarithmic. This target bit allocation is based on Shannon's rate-distortion theory, which deals with the minimization of source distortion subject to a channel rate constraint. Simulation results show that the proposed target bit allocation algorithm provides better performance than the one in MPEG-2 Test Model 5 (TM5).

This paper describes two different S/W-based video codec systems. One is a frame-based video codec with fixed structure based on ITU H.263 standard and the other an object-based video codec with flexible architecture based on ISO MPEG-4 standard currently under specification and planned to be finalized in 1998. These codes are an experimental implementations for examining the feasibility of real-time and/or flexible S/W-based video codecs operating in intra and/or internetworking environments.

The video transmission for real-time viewing over the Internet is a core operation for the multimedia services. However, its realization is very difficult because the Internet has two major problems, namely, very narrow endpoint-bandwidth and the network jitter. We already proposed a scalable video transmission method in [8] which used MPEG-4 video VM(Verification Model) 2.0[3] for very low bit rate coding and an adaptive temporal rate control of video objects to overcome the network jitter problem. In this paper, we present the improved adaptive temporal rate control scheme for the scalable transmission. Experimental results for three test video sequences show that the adaptive temporal rate control can transfer the video bitstream at source frame rate under variable network condition.

For the purpose of a flexible coded video transmission over a heterogeneous network, we propose a new packetization method for coded video data. The proposed method achieves small degradation of coded picture quality in case of packet discard at the network node and does not require heavy processing load for bitrate control operation. Computer simulation results show that the bitrate reduction from 384 kb/s to 192 kb/s does not cause severe degradation in picture quality.

Innovation on digital compression technology, high speed networking technology, digital broadcasting technology and distributed processing technology provides new interactive multimedia service that integrates telecommunication, broadcasting, and computer application. To meet the new trends, we focus our research direction to integrate the telecommunication services and the broadcasting services. For this purpose, we designed and implemented an interactive multimedia service platform based on the DAVIC specification. Our research results in the development of integrated interactive services and its experience will be feedback to the field trial service that will be launched from next year. In this paper, we will describe the overall architecture and system implementation results of the interactive multimedia service platform.

For constructing an intelligent multimedia database system for medical diagnosis, we are focusing on two technological points. One is a retrieval algorithm of databases, and the other is a coding algorithm of multimedia contents. For the first, previously we proposed a front-end database preprocessor called“keyword-network”, and in this paper we present its extended model providing an intelligent logical AND searching function especially for medical differential diagnosis. For the second, we present examples of multimedia intellectual coding methods for cardiovascular examination records.

This Paper describes the system configuration and implementation method of an IMPRES (Interactive Multimedia exPRESS) terminal. The terminal is developed in the form of a PC add-on board that is called MINIBA (Media and Network Interface Board Assembly). IMPRESS is a communication service platform that support various interactive multimedia services. The main objectives of developing IMPRESS are to implement DAVIC compliant services platform to verify DAVIC specification and to support testing environment. Currently IMPRESS terminal provides MPEG-2 quality movies on demand, switched video broadcasting, and home-shopping service to the service user, based on the ATM network.

A new DTPP (Desk-Top Program Production) system has been developed that enables multiple program producers (directors) working at different locations to collaborate over a computer network and prepare a single program for broadcasting. In this system, information is shared among users by exchanging data edited on non-linear editing terminals in program post-production work over a network in real time. In short, the new DTPP system provides a collaborative work space for producing TV programs. The system does not make use of a special server for collaborative work but rather multiple interconnected editing terminals having the same functions. In this configuration, data at a terminal which has just been edited by some operation is forwarded to all other connected terminals for updating. This form of information sharing, however, requires that some sort of data synchronizing method be established since multiple terminals are operating on the same data simultaneously. We therefore adopt a method whereby the system synchronizes the clocks on each terminal at the time of connection and sends an operation time stamp together with edited data. This enables most recently modified data to be identified and all information on all terminals to be updated appropriately. This paper provides an overview of this new collaborative DTPP system and describes the techniques for exchanging edited data and synchronizing data.

In this study, we focus on precise and natural cooperative object manipulation in a virtual space. We introduce two virtually expanded physical laws-virtual mechanical equilibrium on a rigid object and exclusive object arrangement-to create realistic cooperative manipulation. We have built a trial system according to our proposed design. The method is expected to allow users to exchange intended manipulation by haptic and visual channels.

In this paper, we present a VR-based interface method which provides users more natural, realistic, and interactive communication and collaboration tool. Since most services in the communication systems matches with the services in the real world, the best understanding would be achieved when the communication services are represented in accordance with the services in the real world. However, conventional text-based interface and 2D GUI cannot provide such reality to the users. In this paper, we discuss VR-based interface to overcome such difficulty and introduce one instance of communication system using the VR-based realistic, what we call Virtual Village, which we are currently developing. This application might be applied to education in virtual space, desktop conferencing system, and entertainment such as MUD or the games in the distributed environment, etc.

In this paper, we report a system specifications of Hi-resolution and four-screen stereo image projection system which was established in VBL(Venture Business Laboratory) at Tokyo Institute of Technology on 3rd July 1996. The system is 3m $\times$ 3m $\times$2.2m rectangular parallelepiped composed of three 150inch side screens and floor. Host computers can generate virtual environment s in real-time and four projecters project these images to the screens. Viewers are, therefore, surrounded by these screens and an illusion of immersion can be created. Because, the views of users are completely covered with the projection images and many kinds of interactive devices can be used in this system. Moreover, many users can have experience the virtual environments at the same time. Usually, this kind of system uses hi-performance graphics workstations for host computers. One is SGI(Silicon Graphics, Inc.) Onyx with 3 Reality Engines. The other system is 4 personal computers. Because hi-performance and low price graphics accelerators for personal computer have been developed in these years, the abilities of VR(Virtual Reality) systems based on personal computers should be investigated.

A method is proposed to generalize the image transformation from an image to another one according to a pair of example images. When an original image and its target image are given, the unknown image transformation from the original image to the target one in automatically approximated by a sequence of several known image transformation filters by the method. The target image is assumed to be generated manually by using a drawing software. In this method, the order of image transformation filers is regarded as the chromosome of a virtual living thing and is evolved according to Genetic Algorithm. This method can be applied to automatic construction of expert systems for image processing.

This paper proposes the regularized constrained iterative image restoration algorithms which apply new space-adaptive methods to degraded image signals, and analyzes the convergence condition of the proposed algorithm. First, we introduce space-adaptive regularization operators which change according to edge characteristics of local images in order to effectively prevent the restored edges and boundaries from reblurring. And, pseudo projection operator is used to reduce the ringing artifact which results from extensive amplification of noise components in the restoration process. The analysed algorithm is stable convergent to the fixed point. According to the experimental results for various signal-to-noise ratios(SNR) and blur models, the proposed algorithms other methods and is robust to noise effects and edge reblurring by regularization especially.

This paper proposes a object tracking method based on the local moments, or moment based on the local moments, or moment of some restricted area, in which the idea of the viewpoint and the visual filed corresponding to the local area of an image is introduced. Using local moment with the optimally controlled viewpoint and visual field, the target position and its breadth are estimated robustly. By two experiments, the validity of the proposed method is shown.

We improved a visualization system for a player's sequential forms called‘Multi-Motion’which has ever been used in NHK live broadcasting. It is to extract only the player's forms from background, and overlap them into one frame sequentially after fast performance for a vault game of gymnastics. It helps TV viewers understand the player's performing forms easily. We increased the performance of Multi-Motion by shortening the distance between the current frame and the reference frame, and the synthesis of the reference frame.

In this paper we present an algorithm to generate new views of a scene, starting with images from weakly calibrated cameras. Errors in 3D scene reconstruction usually gets reflected in the quality of the new scene generated, so we seek a direct method for reprojection. In this paper, we use the knowledge of dense point matches and their affine coordinate values to estimate the corresponding affine coordinate values in the new scene. We borrow ideas from the object recognition literature, and extend them significantly to solve the problem of reprojection. Unlike the epipolar line intersection algorithms for reprojection which requires at least eight matched points across three images, we need only five matched points. The theory of reprojection is used with hardware based rendering to achieve fast rendering. We demonstrate our results of novel view generation from stereopairs for arbitrary locations of the virtual camera.

We present a method to estimate a dense and sharp depth map using multiple cameras for the application to flexible video production. A key issue for obtaining sharp depth map is how to overcome the harmful influence of occlusion. Thus, we first propose to selectively use the depth information from multiple cameras. With a simple sort and discard technique, we resolve the occlusion problem considerably at a slight sacrifice of noise tolerance. However, boundary overreach of more textured area to less textured area at object boundaries still remains to be solved. We observed that the amount of boundary overreach is less than half the size of the matching window and, unlike usual stereo matching, the boundary overreach with the proposed occlusion-overcoming method shows very abrupt transition. Based on these observations, we propose a hierarchical estimation scheme that attempts to reduce boundary overreach such that edges of the depth map coincide with object boundaries on the one hand, and to reduce noisy estimates due to insufficient size of matching window on the other hand. We show the hierarchical method can produce a sharp depth map for a variety of images.

One of the inherent problems in constructing the sign language dictionary is how to make retrieval and comparison operations on the visual database of signs. This paper describes a retrieval method, especifically for Japanese signs. This method has a useful capability for flexible retrieval of the sign from a bilingual dictionary. Our method can retrieve similar signs to the given input. The retrieval mechanism is essentially based on similarity between the given verbal description and verbal descriptions in a retrieval database. The similarity measure of verbal descriptions can be considered as the approximations for the similarity of sign motion images. As a results of our experiment, the success ratio of the retrievals is 96% in averages.