Methodology for Identifying Issues of User Reviews from the Perspective of Evaluation Criteria: Focus on a Hotel Information Site

As a result of the growth of Internet data and the rapid development of Internet technology, "big data" analysis has gained prominence as a major approach for evaluating and mining enormous data for various purposes. Especially, in recent years, people tend to share their experiences related to their leisure activities while also reviewing others' inputs concerning their activities. Therefore, by referring to others' leisure activity-related experiences, they are able to gather information that might guarantee them better leisure activities in the future. This phenomenon has appeared throughout many aspects of leisure activities such as movies, traveling, accommodation, and dining. Apart from blogs and social networking sites, many other websites provide a wealth of information related to leisure activities. Most of these websites provide information of each product in various formats depending on different purposes and perspectives. Generally, most of the websites provide the average ratings and detailed reviews of users who actually used products/services, and these ratings and reviews can actually support the decision of potential customers in purchasing the same products/services. However, the existing websites offering information on leisure activities only provide the rating and review based on one stage of a set of evaluation criteria. Therefore, to identify the main issue for each evaluation criterion as well as the characteristics of specific elements comprising each criterion, users have to read a large number of reviews. In particular, as most of the users search for the characteristics of the detailed elements for one or more specific evaluation criteria based on their priorities, they must spend a great deal of time and effort to obtain the desired information by reading more reviews and understanding the contents of such reviews. Although some websites break down the evaluation criteria and direct the user to input their reviews according to different levels of criteria, there exist excessive amounts of input sections that make the whole process inconvenient for the users. Further, problems may arise if a user does not follow the instructions for the input sections or fill in the wrong input sections. Finally, treating the evaluation criteria breakdown as a realistic alternative is difficult, because identifying all the detailed criteria for each evaluation criterion is a challenging task. For example, if a review about a certain hotel has been written, people tend to only write one-stage reviews for various components such as accessibility, rooms, services, or food. These might be the reviews for most frequently asked questions, such as distance between the nearest subway station or condition of the bathroom, but they still lack detailed information for these questions. In addition, in case a breakdown of the evaluation criteria was provided along with various input sections, the user might only fill in the evaluation criterion for accessibility or fill in the wrong information such as information regarding rooms in the evaluation criteria for accessibility. Thus, the reliability of the segmented review will be greatly reduced. In this study, we propose an approach to overcome the limitations of the existing leisure activity information websites, namely, (1) the reliability of reviews for each evaluation criteria and (2) the difficulty of identifying the detailed contents that make up the evaluation criteria. In our proposed methodology, we first identify the review content and construct the lexicon for each evaluation criterion by using the terms that are frequently used for each criterion. Next, the sentences in the review documents containing the terms in the constructed lexicon are decomposed into review units, which are then reconstructed by using the evaluation criteria. Finally, the issues of the constructed review units by evaluation criteria are derived and the summary results are provided. Apart from the derived issues, the review units are also provided. Therefore, this approach aims to help users save on time and effort, because they will only be reading the relevant information they need for each evaluation criterion rather than go through the entire text of review. Our proposed methodology is based on the topic modeling, which is being actively used in text analysis. The review is decomposed into sentence units rather than considering the whole review as a document unit. After being decomposed into individual review units, the review units are reorganized according to each evaluation criterion and then used in the subsequent analysis. This work largely differs from the existing topic modeling-based studies. In this paper, we collected 423 reviews from hotel information websites and decomposed these reviews into 4,860 review units. We then reorganized the review units according to six different evaluation criteria. By applying these review units in our methodology, the analysis results can be introduced, and the utility of proposed methodology can be demonstrated.

사용자 리뷰의 평가기준 별 이슈 식별 방법론: 호텔 리뷰 사이트를 중심으로

최근 IT기술의 발전에 따라 많은 사람들이 자신들의 여가활동에 대한 경험을 공유하고 있으며, 역으로 다른 사람들의 여가활동에 대한 경험을 참고하여 더 나은 여가활동을 누릴 수 있는 기회를 얻게 되었다. 이러한 현상은 영화, 숙박, 음식, 여행 등 여가활동 전반에 걸쳐 나타나고 있으며, 그 중심에는 여가활동에 대한 정보를 요약하여 제공하는 수많은 사이트가 있다. 대부분의 여가활동 정보 사이트는 각 상품에 대한 평균 평점뿐만 아니라 상세 리뷰를 제공함으로써, 해당 상품을 구매하고자 하는 잠재고객의 의사결정을 지원하고 있다. 하지만 기존 대부분의 사이트는 한 단계의 평가기준에 따라 평점과 리뷰를 제공하기 때문에, 각 평가기준을 구성하는 세부요소에 대한 특징과 평가기준 별 주요 이슈를 파악하기 위해서는 상당히 많은 수의 리뷰를 직접 읽어야 한다는 불편이 따른다. 즉 사용자는 자신이 중요한 것으로 생각하는 평가기준에 대한 조건을 파악하기 위해, 많은 수의 리뷰를 하나하나 읽어보는 과정에서 많은 시간과 노력을 소비하게 된다. 예를 들어 호텔의 접근성, 객실, 서비스, 음식 등 한 단계의 평가기준만을 사용하여 평점과 리뷰를 제공하는 사이트의 경우, 접근성 중 특히 지하철역과의 거리, 객실 중 특히 욕실의 상태를 살펴보고자 하는 사용자에게 필요한 정보를 충분히 제공하지 못하게 된다. 따라서 본 연구에서는 기존 여가활동 정보 사이트의 한계, 즉 평가기준별로 입력된 리뷰를 신뢰하기 어렵다는 점과 평가기준을 구성하고 있는 세부 내용을 파악하기 어렵다는 점을 극복하기 위한 방안을 제시하고자 한다. 본 연구에서 제안하는 방법론은 사용자가 별도의 구분 없이 입력한 리뷰를 그 내용에 따라 평가기준별로 자동 분류하고, 각 평가 기준 별 주요 이슈를 요약하여 제공한다. 제안 방법론은 최근 텍스트 분석에 활발하게 사용되고 있는 토픽 모델링(Topic Modeling)에 기반을 두고 있으며, 각 리뷰를 하나의 문서 단위로 사용하는 것이 아니라 리뷰를 문장 단위로 끊어 개별 리뷰 유닛(Review Unit)으로 분해한 뒤, 평가기준별로 리뷰 유닛을 재구성하여 분석한다는 측면에서 기존의 토픽 모델링 기반 연구와 큰 차이가 있다고 할 수 있다. 본 논문에서는 제안 방법론을 실제 호텔 정보 사이트에서 수집한 423건의 리뷰 문서에 적용하여 6가지 평가기준에 대해 총 4,860건의 리뷰 유닛을 재구성하고, 이에 대한 분석 결과를 소개함으로써 제안 방법론의 유용성을 간접적으로 보인다.