• Journal of Information Processing Systems
• /
• v.17 no.2
• /
• pp.337-351
• /
• 2021

Automatically recognizing facial expressions in video sequences is a challenging task because there is little direct correlation between facial features and subjective emotions in video. To overcome the problem, a video facial expression recognition method using spatiotemporal recurrent neural network and feature fusion is proposed. Firstly, the video is preprocessed. Then, the double-layer cascade structure is used to detect a face in a video image. In addition, two deep convolutional neural networks are used to extract the time-domain and airspace facial features in the video. The spatial convolutional neural network is used to extract the spatial information features from each frame of the static expression images in the video. The temporal convolutional neural network is used to extract the dynamic information features from the optical flow information from multiple frames of expression images in the video. A multiplication fusion is performed with the spatiotemporal features learned by the two deep convolutional neural networks. Finally, the fused features are input to the support vector machine to realize the facial expression classification task. The experimental results on cNTERFACE, RML, and AFEW6.0 datasets show that the recognition rates obtained by the proposed method are as high as 88.67%, 70.32%, and 63.84%, respectively. Comparative experiments show that the proposed method obtains higher recognition accuracy than other recently reported methods.

• 한국컴퓨터산업교육학회:학술대회논문집
• /
• 2003.11a
• /
• pp.127-133
• /
• 2003

The data models which express all types of video information physically and logically. and the definition of spatiotemporal relationship of video data objects In This paper, we classifies meta-model for efficient management on spatiotemporal relationship between two objects in video image data, suggests meta-models based on Rambaugh's OMT technique, and expanded user model to apply the adaptive model, established from hyper-media or web agent to VOD. The proposed meta-model uses data's special physical feature: the effects of camera's and editing effects of shot, and 17 spatial relations on Allen's 13 temporal relations, topology and direction to include logical presentation of spatiotemporal relation for possible spatiotemporal reference and having unspecified applied mediocrity.

• International journal of advanced smart convergence
• /
• v.12 no.4
• /
• pp.88-97
• /
• 2023

Traffic flow prediction is of great significance in urban planning and traffic management. As the complexity of urban traffic increases, existing prediction methods still face challenges, especially for the fusion of spatiotemporal information and the capture of long-term dependencies. This study aims to use the fusion model of graph neural network to solve the spatio-temporal information fusion problem in traffic flow prediction. We propose a new deep learning model Spatio-Temporal Information Fusion using Graph Neural Networks (STFGNN). We use GCN module, TCN module and LSTM module alternately to carry out spatiotemporal information fusion. GCN and multi-core TCN capture the temporal and spatial dependencies of traffic flow respectively, and LSTM connects multiple fusion modules to carry out spatiotemporal information fusion. In the experimental evaluation of real traffic flow data, STFGNN showed better performance than other models.

• Journal of the Korea Institute of Information and Communication Engineering
• /
• v.14 no.9
• /
• pp.2057-2064
• /
• 2010

With the development of wireless network technology, the location information of a spatiotemporal object which changes their location is used in various application. Each spatiotemporal object has many location information, hence it is inefficient to search all trajectory information of spatiotemporal objects for a range query. In this paper, we propose an efficient method which divides a trajectory and stores its division data on restricted storage space. Using suboptimal split algorithm, an extended split algorithm that minimizes the volume of EMBRs(Extended Minimum Bounding Box) is designed and simulated. Our experimental evaluation confirms the effectiveness and efficiency of our proposed splitting policy

• Journal of the Korea Institute of Information and Communication Engineering
• /
• v.15 no.10
• /
• pp.2188-2194
• /
• 2011

With the development of wireless network technology, Storing the location information of a spatiotemporal object was very necessary. Each spatiotemporal object has many unnecessariness location information, hence it is inefficient to search all trajectory information of spatiotemporal objects. In this paper, we propose an efficient method which increase searching efficiency. Using EMBR(Extend Minimun Bounding Rectangle), an LinearMarge split algorithm that minimizes the volume of MBRs is designed and simulated. Our experimental evaluation confirms the effectiveness and efficiency of our proposed splitting policy.

• ETRI Journal
• /
• v.43 no.3
• /
• pp.538-548
• /
• 2021

This paper proposes an approach to improve the performance of no-reference video quality assessment for sports videos with dynamic motion scenes using an efficient spatiotemporal model. In the proposed method, we divide the video sequences into video blocks and apply a 3D shearlet transform that can efficiently extract primary spatiotemporal features to capture dynamic natural motion scene statistics from the incoming video blocks. The concatenation of a deep residual bidirectional gated recurrent neural network and logistic regression is used to learn the spatiotemporal correlation more robustly and predict the perceptual quality score. In addition, conditional video block-wise constraints are incorporated into the objective function to improve quality estimation performance for the entire video. The experimental results show that the proposed method extracts spatiotemporal motion information more effectively and predicts the video quality with higher accuracy than the conventional no-reference video quality assessment methods.

• Journal of Institute of Control, Robotics and Systems
• /
• v.15 no.1
• /
• pp.111-119
• /
• 2009

For daily life interaction with human, robots need the capability of encoding and storing cognitive information and retrieving it contextually. In this paper, spatiotemporal grounding of cognitive information for a language based cognitive system is presented. The cognitive information of the event occurred at a robot is described with a sentence, stored in a memory, and retrieved contextually. Each sentence is parsed, discriminated with the functional type of it, and analyzed with argument structure for connecting to cognitive information. With the proposed grounding, the cognitive information is encoded to sentence form and stored in sentence memory with object descriptor. Sentences are retrieved for answering questions of human by searching temporal information from the sentence memory and doing spatial reasoning in schematic imagery. An experiment shows the feasibility and efficiency of the spatiotemporal grounding for advanced service robot.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.12 no.4
• /
• pp.1618-1637
• /
• 2018

By means of compressive sensing (CS) technique, this paper considers the collection of sensor data with spatiotemporal correlations in wireless sensor networks (WSNs). In energy-constrained WSNs, one-dimensional CS methods need a lot of data transmissions since they are less applicable in fully exploiting the spatiotemporal correlations, while the Kronecker CS (KCS) methods suffer performance degradations when the signal dimension increases. In this paper, an appropriate sensing matrix as well as an efficient sensing method is proposed to further reduce the data transmissions without the loss of the recovery performance. Different matrices for the temporal signal of each sensor node are separately designed. The corresponding energy-efficient data gathering method is presented, which only transmitting a subset of sensor readings to recover data of the entire WSN. Theoretical analysis indicates that the sensing structure could have the relatively small mutual coherence according to the selection of matrix. Compared with the existing spatiotemporal CS (CS-ST) method, the simulation results show that the proposed efficient measurement method could reduce data transmissions by about 25% with the similar recovery performance. In addition, compared with the conventional KCS method, for 95% successful recovery, the proposed sensing structure could improve the recovery performance by about 20%.

• The KIPS Transactions:PartD
• /
• v.9D no.1
• /
• pp.43-56
• /
• 2002

Most of works have been performed on representation of spatiotemporal objects from various points of view. Most of them represent spatiotemporal objects using approaches from GIS, temporal databases, object-oriented databases or data type. Spatiotemporal objects can be classified as objects whose position and shape changes discretely over time, objects whose position changes continuously and objects whose shape changes continuously as well as position. Previous works on spatiotemporal model have focused on only one of them. In this paper, we propose a spatiotemporal model that can represent three types of objects in Euclidean plan. For this purpose, we represent both discrete and continuous moving objects by defining temporal model extended from valid time and by defining relationship between two consecutive versions of objects. The proposed spatiotemporal object model is based on open GIS specification so that it has compatibility with existing spatial data model.

• Journal of the Institute of Electronics Engineers of Korea CI
• /
• v.38 no.6
• /
• pp.1-17
• /
• 2001

Nowadays, spatiotemporal data models deal with objects which can be potentially useful for wide range applications in order to describe complex objects with spatial and/or temporal facilities. However, the information needed by each application usually varies, specially in the geographic information which depends on the kind of time oriented views, as defined in the modeling phase of the spatiotemporal geographic data design. To be able to deal with such diverse needs, geographic information systems must offer features that manipulate geometric, space-dependent(i.e, thematic), and spatial relationship positions with multiple time oriented views. This paper addresses problems of the formal definition of relationships among spatiotemporal objects and their properties on geographic information systems. The geographical data are divided in two main classes : geo-objects and geo-fields, which describe discrete and continuous representations of the spatial reality. I study semantics and syntax about the temporal changes of attributes and the relationship roles on geo-objects and non-geo-objects, This result will contribute on the design of object oriented spatiotemporal data model which is distinguishied from the recent geographic information system of the homogeneously anchored spatial objects