Aspect-Based Sentiment Analysis Using BERT: Developing Aspect Category Sentiment Classification Models (BERT를 활용한 속성기반 감성분석: 속성카테고리 감성분류 모델 개발)
-
- Journal of Intelligence and Information Systems
- /
- 제26권4호
- /
- pp.1-25
- /
- 2020
Sentiment Analysis (SA) is a Natural Language Processing (NLP) task that analyzes the sentiments consumers or the public feel about an arbitrary object from written texts. Furthermore, Aspect-Based Sentiment Analysis (ABSA) is a fine-grained analysis of the sentiments towards each aspect of an object. Since having a more practical value in terms of business, ABSA is drawing attention from both academic and industrial organizations. When there is a review that says "The restaurant is expensive but the food is really fantastic", for example, the general SA evaluates the overall sentiment towards the 'restaurant' as 'positive', while ABSA identifies the restaurant's aspect 'price' as 'negative' and 'food' aspect as 'positive'. Thus, ABSA enables a more specific and effective marketing strategy. In order to perform ABSA, it is necessary to identify what are the aspect terms or aspect categories included in the text, and judge the sentiments towards them. Accordingly, there exist four main areas in ABSA; aspect term extraction, aspect category detection, Aspect Term Sentiment Classification (ATSC), and Aspect Category Sentiment Classification (ACSC). It is usually conducted by extracting aspect terms and then performing ATSC to analyze sentiments for the given aspect terms, or by extracting aspect categories and then performing ACSC to analyze sentiments for the given aspect category. Here, an aspect category is expressed in one or more aspect terms, or indirectly inferred by other words. In the preceding example sentence, 'price' and 'food' are both aspect categories, and the aspect category 'food' is expressed by the aspect term 'food' included in the review. If the review sentence includes 'pasta', 'steak', or 'grilled chicken special', these can all be aspect terms for the aspect category 'food'. As such, an aspect category referred to by one or more specific aspect terms is called an explicit aspect. On the other hand, the aspect category like 'price', which does not have any specific aspect terms but can be indirectly guessed with an emotional word 'expensive,' is called an implicit aspect. So far, the 'aspect category' has been used to avoid confusion about 'aspect term'. From now on, we will consider 'aspect category' and 'aspect' as the same concept and use the word 'aspect' more for convenience. And one thing to note is that ATSC analyzes the sentiment towards given aspect terms, so it deals only with explicit aspects, and ACSC treats not only explicit aspects but also implicit aspects. This study seeks to find answers to the following issues ignored in the previous studies when applying the BERT pre-trained language model to ACSC and derives superior ACSC models. First, is it more effective to reflect the output vector of tokens for aspect categories than to use only the final output vector of [CLS] token as a classification vector? Second, is there any performance difference between QA (Question Answering) and NLI (Natural Language Inference) types in the sentence-pair configuration of input data? Third, is there any performance difference according to the order of sentence including aspect category in the QA or NLI type sentence-pair configuration of input data? To achieve these research objectives, we implemented 12 ACSC models and conducted experiments on 4 English benchmark datasets. As a result, ACSC models that provide performance beyond the existing studies without expanding the training dataset were derived. In addition, it was found that it is more effective to reflect the output vector of the aspect category token than to use only the output vector for the [CLS] token as a classification vector. It was also found that QA type input generally provides better performance than NLI, and the order of the sentence with the aspect category in QA type is irrelevant with performance. There may be some differences depending on the characteristics of the dataset, but when using NLI type sentence-pair input, placing the sentence containing the aspect category second seems to provide better performance. The new methodology for designing the ACSC model used in this study could be similarly applied to other studies such as ATSC.
As having the movement of developing private brand (PB) goods, domestic big retailers are facing up with new problems. Thus, it is required studies of PB products, and how consumers recognize PB products as a consideration commodity set. Also, it is worthy in order that it gives us the important meaning on the marketing strategy with focusing on evaluating the differences between customers buying PB grocery goods with respect to demographic characteristics and purchasing behaviors. PB has some advantages for customers and retailers. However, according to AC Nielson's report (2005), Asian and emerging market has 1/5 sales relatively to Western countries. But we can assume that the emerging market has the most potential growth through this result. As a result from several other studies, it becomes necessary to not only increase the rate of selling composition of PB product temporarily, but also analyze the characteristics of customers using big retailers and segmenting customer groups to make PB product as a consideration commodity set for them. In addition, it is needed to have a variety of acts of marketing. From studies related to PB, there is a prejudice - cheap products have low quality - but, evaluation by customers who have used those products shows neutral stand, and there is a study representing that it is the most important to accumulate the belief between the retailers selling PB products and consumers using those for the accurate evaluation and intention on purchasing. Also, by the result from analyzing the characteristics of customers buying PB products, we could assume that higher income and higher education level, more preference on PB products. Especially, according to TNS's research, the primary targets of PB product are 30's who seeks value for money and planned spending habits, and 40's who have teenager children, and are interested in encouraging themselves. This paper used Probit model to analyze the characteristics of consumers. This model helps us to analyze with the variables representing the demographic characteristics of consumers (gender, age, educational level, occupation, income level, living area), and variables related to purchasing behavior (visiting frequency on big retailers, the average amount that they pay for goods in there, and check-up which brand made those goods). The method we used in this study is by man to man interview and survey on-line with the rate of 89% and 11% in Seoul and Gyunggi Province, respectively, for about one month from the beginning of February, 2008. As a result of this, under the assumption that people buy PB products more as long as they go shopping more, it was not meaningful for target groups which we pointed out as frequently visiting customers to be. Although, we have expected women buy more PB products than men do, gender doesn't mean anything for the result. And, it has inferred that married people buy more PB goods than singles do. It was also meaningless with variables related to occupation. Because housewives are often exposed to any kind of supermarket than workers are, we could not get any relatives. Moreover, we couldn't proof that younger generation prefer big retailers more than older people who 50~60's. Education levels doesn't affect on the purchase of PB product as well. Related to living area, the result is statistically not similar as we expected whether living in Seoul or not. It shows there is no relationship with the preference on retail brands and PB products, and it is similar with the study researched by TNS(2008) that customers tend to buy PB product impulsively no matter which brand it is and where they are even though their shopping place is the big market where customers are often using. Variables on which we had meaningful results are income level and living place. That is, customers who have 3,000,000~6,000,000 WON every month on average are more willing to buy PB products than other customers whose income is over 6,000,000 WON, and residents not living in Seoul prefer PB goods than those who are living in Seoul. To explain more about what we got, if there is only one condition about customer's visiting frequency on big retails, we could come up with this result that more exposed to PB products, more purchasing frequency. Consequently, it brings the important insight that large retailers have to prepare something to make customers visit them often to increase selling rate of PB products. To demonstrate the result of analyzing more, what is more efficient variables are demographically including marital status, income level, and residential area to buy items that affect the PB products and could include the frequency of visiting large markets by the purchase habits. Specifically, then, married couples rather than singles, middle-income customers than high-income customers, and local residents not living in Seoul than customers in Seoul are more likely to purchase PB goods. In addition, as long as a customer visits two times more, then the purchasing rate of PB products is to increase over 5.3%. Therefore, it seems that retailers are better to make a shopping place as fun and comfortable places. With overwhelming the idea that PB products are just cheap, one-time purchase goods, it is needed to increase the loyalty on those goods like NB products, try to make PB products as a consideration products set, and occur to sustainable sales. Especially, as suggested by this paper, it seems like it strongly needs to identify the characteristics of customers who prefer PB, to segment those customers, and to select the main target, and to do positioning with well-planned marketing strategies. Then, it is able to give us a meaningful point on marketing strategy by developing the field of PB study, identifying the difference of life style and shopping habits of customers.
Recently banks and large financial institutions have introduced lots of Robo-Advisor products. Robo-Advisor is a Robot to produce the optimal asset allocation portfolio for investors by using the financial engineering algorithms without any human intervention. Since the first introduction in Wall Street in 2008, the market size has grown to 60 billion dollars and is expected to expand to 2,000 billion dollars by 2020. Since Robo-Advisor algorithms suggest asset allocation output to investors, mathematical or statistical asset allocation strategies are applied. Mean variance optimization model developed by Markowitz is the typical asset allocation model. The model is a simple but quite intuitive portfolio strategy. For example, assets are allocated in order to minimize the risk on the portfolio while maximizing the expected return on the portfolio using optimization techniques. Despite its theoretical background, both academics and practitioners find that the standard mean variance optimization portfolio is very sensitive to the expected returns calculated by past price data. Corner solutions are often found to be allocated only to a few assets. The Black-Litterman Optimization model overcomes these problems by choosing a neutral Capital Asset Pricing Model equilibrium point. Implied equilibrium returns of each asset are derived from equilibrium market portfolio through reverse optimization. The Black-Litterman model uses a Bayesian approach to combine the subjective views on the price forecast of one or more assets with implied equilibrium returns, resulting a new estimates of risk and expected returns. These new estimates can produce optimal portfolio by the well-known Markowitz mean-variance optimization algorithm. If the investor does not have any views on his asset classes, the Black-Litterman optimization model produce the same portfolio as the market portfolio. What if the subjective views are incorrect? A survey on reports of stocks performance recommended by securities analysts show very poor results. Therefore the incorrect views combined with implied equilibrium returns may produce very poor portfolio output to the Black-Litterman model users. This paper suggests an objective investor views model based on Support Vector Machines(SVM), which have showed good performance results in stock price forecasting. SVM is a discriminative classifier defined by a separating hyper plane. The linear, radial basis and polynomial kernel functions are used to learn the hyper planes. Input variables for the SVM are returns, standard deviations, Stochastics %K and price parity degree for each asset class. SVM output returns expected stock price movements and their probabilities, which are used as input variables in the intelligent views model. The stock price movements are categorized by three phases; down, neutral and up. The expected stock returns make P matrix and their probability results are used in Q matrix. Implied equilibrium returns vector is combined with the intelligent views matrix, resulting the Black-Litterman optimal portfolio. For comparisons, Markowitz mean-variance optimization model and risk parity model are used. The value weighted market portfolio and equal weighted market portfolio are used as benchmark indexes. We collect the 8 KOSPI 200 sector indexes from January 2008 to December 2018 including 132 monthly index values. Training period is from 2008 to 2015 and testing period is from 2016 to 2018. Our suggested intelligent view model combined with implied equilibrium returns produced the optimal Black-Litterman portfolio. The out of sample period portfolio showed better performance compared with the well-known Markowitz mean-variance optimization portfolio, risk parity portfolio and market portfolio. The total return from 3 year-period Black-Litterman portfolio records 6.4%, which is the highest value. The maximum draw down is -20.8%, which is also the lowest value. Sharpe Ratio shows the highest value, 0.17. It measures the return to risk ratio. Overall, our suggested view model shows the possibility of replacing subjective analysts's views with objective view model for practitioners to apply the Robo-Advisor asset allocation algorithms in the real trading fields.