The Intelligent Determination Model of Audience Emotion for Implementing Personalized Exhibition

Recently, due to the introduction of high-tech equipment in interactive exhibits, many people's attention has been concentrated on Interactive exhibits that can double the exhibition effect through the interaction with the audience. In addition, it is also possible to measure a variety of audience reaction in the interactive exhibition. Among various audience reactions, this research uses the change of the facial features that can be collected in an interactive exhibition space. This research develops an artificial neural network-based prediction model to predict the response of the audience by measuring the change of the facial features when the audience is given stimulation from the non-excited state. To present the emotion state of the audience, this research uses a Valence-Arousal model. So, this research suggests an overall framework composed of the following six steps. The first step is a step of collecting data for modeling. The data was collected from people participated in the 2012 Seoul DMC Culture Open, and the collected data was used for the experiments. The second step extracts 64 facial features from the collected data and compensates the facial feature values. The third step generates independent and dependent variables of an artificial neural network model. The fourth step extracts the independent variable that affects the dependent variable using the statistical technique. The fifth step builds an artificial neural network model and performs a learning process using train set and test set. Finally the last sixth step is to validate the prediction performance of artificial neural network model using the validation data set. The proposed model is compared with statistical predictive model to see whether it had better performance or not. As a result, although the data set in this experiment had much noise, the proposed model showed better results when the model was compared with multiple regression analysis model. If the prediction model of audience reaction was used in the real exhibition, it will be able to provide countermeasures and services appropriate to the audience's reaction viewing the exhibits. Specifically, if the arousal of audience about Exhibits is low, Action to increase arousal of the audience will be taken. For instance, we recommend the audience another preferred contents or using a light or sound to focus on these exhibits. In other words, when planning future exhibitions, planning the exhibition to satisfy various audience preferences would be possible. And it is expected to foster a personalized environment to concentrate on the exhibits. But, the proposed model in this research still shows the low prediction accuracy. The cause is in some parts as follows : First, the data covers diverse visitors of real exhibitions, so it was difficult to control the optimized experimental environment. So, the collected data has much noise, and it would results a lower accuracy. In further research, the data collection will be conducted in a more optimized experimental environment. The further research to increase the accuracy of the predictions of the model will be conducted. Second, using changes of facial expression only is thought to be not enough to extract audience emotions. If facial expression is combined with other responses, such as the sound, audience behavior, it would result a better result.

개인화 전시 서비스 구현을 위한 지능형 관객 감정 판단 모형

최근 기존 전시 공간 내에 유비쿼터스 환경이 구축되면서, 관객과의 상호작용을 통해 전시 효과를 배가할 수 있는 인터랙티브 전시에 많은 사람들의 관심이 집중되고 있다. 이러한 인터랙티브 전시가 보다 고도화되기 위해서는 전시물에 대한 다양한 관객 반응을 측정하고, 이를 통해 대상 관객이 어떤 감정을 느끼는지 예측할 수 있는 적절한 의사결정지원 모형이 요구된다. 이러한 배경에서 본 연구는 인터랙티브 전시 공간 내에서 수집 가능한 다양한 관객 반응 중 얼굴표정의 변화를 이용하여, 관객의 감정을 추론, 판단하는 지능형 모형을 제시한다. 본 연구에서 제시하는 모형은 무자극 상태의 관객의 표정과 자극이 주어졌을 때 관객의 표정이 어떻게 변화하는지 변화량을 측정하여, 이를 기반으로 인공신경망 기법을 이용해 해당 관객의 감정을 판단하는 모형이다. 이 때, 제안모형의 감정 분류체계로는 간결하면서도 실무에 적용이 용이하여 그간 기존 문헌에서 널리 활용되어 온 매력-각성(Valence-Arousal) 모형을 사용한다. 제안모형의 유용성을 검증하기 위해, 본 연구에서는 2011 서울 DMC 컬쳐 오픈 행사에 참여하여, 일반인을 대상으로 얼굴 표정 변화 데이터를 수집하고, 이들이 느끼는 감정 상태를 설문조사하였다. 그리고 나서, 이 자료들을 대상으로 본 연구에서 제안하는 모형을 적용해 보고, 제안모형이 비교모형으로 설정된 통계기반 예측모형에 비해 더 우수한 성과를 보이는지 확인해 보았다. 실험 결과, 본 연구에서 제시하는 모형이 비교 모형인 중회귀분석 모형보다 더 우수한 결과를 제공함을 확인할 수 있었다. 본 연구를 통하여 구축된 관객 감정 판단 모형을 실제 전시장에서 활용한다면 전시물을 관람하는 관객의 반응에 따라 시의적절하면서도 효과적인 대응이 가능하기 때문에, 관객의 몰입과 만족을 보다 증대시킬 수 있을 것으로 기대된다.