A study on the use of a Business Intelligence system : the role of explanations

With the rapid advances in technologies, organizations are more likely to depend on information systems in their decision-making processes. Business Intelligence (BI) systems, in particular, have become a mainstay in dealing with complex problems in an organization, partly because a variety of advanced computational methods from statistics, machine learning, and artificial intelligence can be applied to solve business problems such as demand forecasting. In addition to the ability to analyze past and present trends, these predictive analytics capabilities provide huge value to an organization's ability to respond to change in markets, business risks, and customer trends. While the performance effects of BI system use in organization settings have been studied, it has been little discussed on the use of predictive analytics technologies embedded in BI systems for forecasting tasks. Thus, this study aims to find important factors that can help to take advantage of the benefits of advanced technologies of a BI system. More generally, a BI system can be viewed as an advisor, defined as the one that formulates judgments or recommends alternatives and communicates these to the person in the role of the judge, and the information generated by the BI system as advice that a decision maker (judge) can follow. Thus, we refer to the findings from the advice-giving and advice-taking literature, focusing on the role of explanations of the system in users' advice taking. It has been shown that advice discounting could occur when an advisor's reasoning or evidence justifying the advisor's decision is not available. However, the majority of current BI systems merely provide a number, which may influence decision makers in accepting the advice and inferring the quality of advice. We in this study explore the following key factors that can influence users' advice taking within the setting of a BI system: explanations on how the box-office grosses are predicted, types of advisor, i.e., system (data mining technique) or human-based business advice mechanisms such as prediction markets (aggregated human advice) and human advisors (individual human expert advice), users' evaluations of the provided advice, and individual differences in decision-makers. Each subject performs the following four tasks, by going through a series of display screens on the computer. First, given the information of the given movie such as director and genre, the subjects are asked to predict the opening weekend box office of the movie. Second, in light of the information generated by an advisor, the subjects are asked to adjust their original predictions, if they desire to do so. Third, they are asked to evaluate the value of the given information (e.g., perceived usefulness, trust, satisfaction). Lastly, a short survey is conducted to identify individual differences that may affect advice-taking. The results from the experiment show that subjects are more likely to follow system-generated advice than human advice when the advice is provided with an explanation. When the subjects as system users think the information provided by the system is useful, they are also more likely to take the advice. In addition, individual differences affect advice-taking. The subjects with more expertise on advisors or that tend to agree with others adjust their predictions, following the advice. On the other hand, the subjects with more knowledge on movies are less affected by the advice and their final decisions are close to their original predictions. The advances in predictive analytics of a BI system demonstrate a great potential to support increasingly complex business decisions. This study shows how the designs of a BI system can play a role in influencing users' acceptance of the system-generated advice, and the findings provide valuable insights on how to leverage the advanced predictive analytics of the BI system in an organization's forecasting practices.

비즈니스 인텔리전스 시스템의 활용 방안에 관한 연구: 설명 기능을 중심으로

다양한 빅데이터 기술이 발전함에 따라, 기업의 전략결정에 있어서 과거에는 의사결정자의 직관이나 경험에 의존하는 경향이 있었다면, 현재는 데이터를 활용한 과학적이고 분석적인 접근이 이루어지고 있다. 이에 많은 기업들이 경영정보시스템 중의 하나인 비즈니스 인텔리전스 (Business Intelligence) 시스템의 예측분석 기능을 활용하고 있다. 하지만, 이러한 시스템이 미래의 경영환경 변화를 예측하고 기업의 의사결정을 돕는 조언자 (Advisor)로서 역할을 한다고 가정할 때, 시스템에서 제공하는 분석결과가 의사결정자에게 도움을 주는 조언 (Advice) 의 역할을 하지 못하는 경우가 많은 실정이다. 따라서, 본 연구에서는 미래예측의 문제에 있어 의사결정자가 시스템의 조언을 따르는데 영향을 미치는 요소들과 영향력에 대해 분석하고, 그 결과를 바탕으로 데이터 기반의 의사결정을 보다 적극적으로 지원하는 시스템 환경을 제시하고자 한다. 좀 더 구체적으로는 예측 과정에 대한 자세한 설명이나 근거 제시가 시스템의 예측결과에 대한 의사결정자의 수용정도에 미치는 영향을 연구하였다. 이를 위하여 193명의 실험자를 대상으로 영화의 개봉 주 매출액을 예측하는 업무를 수행하고, 예측에 대한 설명의 길이와 조언자의 유형(사람과 시스템의 조언 비교)뿐 아니라 의사결정자의 개인 특성이 의사결정자의 조언 수용정도에 미치는 영향을 분석하였다. 시스템에서 제공하는 조언 내용인 예측결과와 설명에 대해 의사결정가가 느끼는 유용성, 신뢰성, 만족도가 조언의 수용에 미치는 영향도 분석하였다. 본 연구는 시스템의 분석결과를 조언으로 보고 조언자와 조언에 관한 의사결정학 분야의 선행연구를 접목시켜 경영정보시스템 연구 분야를 확장하였다는 점에서 연구의 의의가 있고, 실무적으로도 데이터 기반의 의사결정을 보다 적극적으로 지원할 수 있는 시스템 환경을 만들기 위해서 고려해야 할 점들을 제시함으로써 시스템 활용을 위한 정책결정에도 도움을 줄 수 있을 것으로 본다.