Information Systems Review (경영정보학연구)

The Korea Society of Management Information Systems (한국경영정보학회)
권호 표지
DOI QR코드

DOI QR Code

Domain Knowledge Incorporated Local Rule-based Explanation for ML-based Bankruptcy Prediction Model

머신러닝 기반 부도예측모형에서 로컬영역의 도메인 지식 통합 규칙 기반 설명 방법

  • Soo Hyun Cho (Department of Big Data Analytics, Ewha Womans University) ;
  • Kyung-shik Shin (School of Business, Ewha Womans University)
  • 조수현 (이화여자대학교 빅데이터분석학) ;
  • 신경식 (이화여자대학교 경영대학)
  • Received : 2021.11.23
  • Accepted : 2022.01.24
  • Published : 2022.02.28
Download PDF
⟨ Previous Next ⟩

Abstract

Thanks to the remarkable success of Artificial Intelligence (A.I.) techniques, a new possibility for its application on the real-world problem has begun. One of the prominent applications is the bankruptcy prediction model as it is often used as a basic knowledge base for credit scoring models in the financial industry. As a result, there has been extensive research on how to improve the prediction accuracy of the model. However, despite its impressive performance, it is difficult to implement machine learning (ML)-based models due to its intrinsic trait of obscurity, especially when the field requires or values an explanation about the result obtained by the model. The financial domain is one of the areas where explanation matters to stakeholders such as domain experts and customers. In this paper, we propose a novel approach to incorporate financial domain knowledge into local rule generation to provide explanations for the bankruptcy prediction model at instance level. The result shows the proposed method successfully selects and classifies the extracted rules based on the feasibility and information they convey to the users.

신용리스크 관리에 해당하는 부도예측모형은 기업에 대한 신용평가라고도 볼 수 있으며 은행을 비롯한 금융기관의 신용평가모형의 기본 지식기반으로 새로운 인공지능 기술을 접목할 수 있는 유망한 분야로 손꼽히고 있다. 고도화된 모형의 실제 응용은 사용자의 수용도가 중요하나 부도예측모형의 경우, 금융전문가 혹은 고객에게 모형의 결과에 대한 설명이 요구되는 분야로 설명력이 없는 모형은 실제로 도입되고 사용자들에게 수용되기에는 어려움이 있다. 결국 모형의 결과에 대한 설명은 모형의 사용자에게 제공되는 것으로 사용자가 납득할 수 있는 설명을 제공하는 것이 모형에 대한 신뢰와 수용을 증진시킬 수 있다. 본 연구에서는 머신러닝 기반 모형에 설명력을 제고하는 방안으로 설명대상 인스턴스에 대하여 로컬영역에서의 설명을 제공하고자 한다. 이를 위해 설명대상의 로컬영역에 유전알고리즘(GA)을 이용하여 가상의 데이터포인트들을 생성한 후, 로컬 대리모델(surrogate model)로 연관규칙 알고리즘을 이용하여 설명대상에 대한 규칙기반 설명(rule-based explanation)을 생성한다. 해석 가능한 로컬 모델의 활용으로 설명을 제공하는 기존의 방법에서 더 나아가 본 연구는 부도예측모형에 이용된 재무변수의 특성을 반영하여 연관규칙으로 도출된 설명에 도메인 지식을 통합한다. 이를 통해 사용자에게 제공되는 규칙의 현실적 가능성(feasibility)을 확보하고 제공되는 설명의 이해와 수용을 제고하고자 한다. 본 연구에서는 대표적인 블랙박스 모형인 인공신경망 기반 부도예측모형을 기반으로 최신의 규칙기반 설명 방법인 Anchor와 비교하였다. 제안하는 방법은 인공신경망 뿐만 아니라 다른 머신러닝 모형에도 적용 가능한 방법(model-agonistic method)이다.

Keywords

Acknowledgement

This work was supported by the National Research Foundation of Korea (NRF-2021R1A2C1012036).

References

  1. Adadi, A. and M. Berrada, "Peeking inside the black-box: A survey on explainable artificial intelligence (XAI)", IEEE Access, Vol.6, 2018, pp. 52138-60.
  2. Agrawal, R., T. Imielinski, and A. Swami, "Mining association rules between sets of items in large databases", ACM SIGMOD Record, Vol.22, No.2, 1993, pp. 207-16.
  3. Bank of Korea, Financial Statement Analysis for 2019, Bank of Korea, 2020.
  4. Carvalho, D. V., E. M. Pereira, and J. S. Cardoso, "Machine learning interpretability: A survey on methods and metrics", Electronics, Vol.8, No.8, 2019, pp. 1-34.
  5. Dash, S., O. Gunluk, and D. Wei, "Boolean decision rules via column generation", Advances in Neural Information Processing System 2018-Decem, 2018, pp. 4655-4665.
  6. Dastile, X., T. Celik, and M. Potsane, "Statistical and machine learning models in credit scoring: A systematic literature survey", Applied Soft Computing Journal, Vol.91, No.106263, 2020, pp. 1-21.
  7. Davidson, W., Financial Statement Analysis Basis For Management Advice, Association of International Certified Professional Accountants, Inc., NC, USA, 2019.
  8. Feng, X., Z. Xiao, B. Zhong, J. Qiu, and Y. Dong, "Dynamic ensemble classification for credit scoring using soft probability", Applied Soft Computing Journal, Vol.65, 2018, pp. 139-51.
  9. Gomez, O., S. Holter, J. Yuan, and E. Bertini, "ViCE: Visual counterfactual explanations for machine learning models", 25th International Conference on Intelligent User Interfaces, 2020.
  10. Grath, R. M., L. Costabello, C. Le Van, P. Sweeney, F. Kamiab, Z. Shen, and F. Lecue, "Interpretable credit application predictions with counterfactual explanations", NIPS 2018 Workshop on Challenges and Opportunities for AI In Financial Services: The Impact of Fairness, Explainability, Accuracy, and Privacy, 2018.
  11. Guidotti, R., A. Monreale, S. Ruggieri, F. Giannotti, D. Pedreschi, and F. Turini, "Factual and counterfactual explanations for black box decision making", IEEE Intelligent Systems, 2019, pp. 14-23.
  12. Guidotti, R., A. Monreale, S. Ruggieri, D. Pedreschi, F. Turini, and F. Giannotti, "Local rule-based explanations of black box decision systems", arXiv preprint arXiv:1805.10820, 2018.
  13. Gunning, D. and D. W. Aha, "DARPA's explainable artificial intelligence program", AI Magazine, Vol.40, No.2, 2019, pp. 44-58.
  14. Hayashi, Y., "Application of a rule extraction algorithm family based on the Re-RX Algorithm to financial credit risk assessment from a pareto optimal perspective", Operations Research Perspectives, Vol.3, 2016, pp. 32-42.
  15. He, H., W. Zhang, and S. Zhang, "A novel ensemble method for credit scoring: Adaption of different imbalance ratios", Expert Systems with Applications, Vol.98, 2018, pp. 105-17.
  16. Helfert, E. A., "Assessment in Business Performance", in Helfert, E. A. (1st ed.), Financial Analysis Tools and Techniques: A Guide for Managers, McGraw-Hill, NY, New York, 2001, pp. 95-160.
  17. Henley, W. E. and D. J. Hand, "A k-nearest-neighbour classifier for assessing consumer credit risk", The Statistician, Vol.45, No.1, 1996, pp. 77-95.
  18. Jardin, P. D., "A two-stage classification technique for bankruptcy prediction", European Journal of Operational Research, Vol.254, No.1, 2016, pp. 236-52.
  19. Lipton, Z. C., "The Mythos of Model Interpretability," 2016 ICML Workshop on Human Interpretability in MachineLearning, 2016
  20. Mahajan, D., C. Tan, and A. Sharma, "Preserving causal constraints in counterfactual explanations for machine learning classifiers", 33rd Conference on Neural Information Processing Systems, 2019.
  21. Marques, A. I., V. Garcia, and J. S. Sanchez, "Two-level classifier ensembles for credit risk assessment", Expert Systems with Applications, Vol.39, No.12, pp. 10916-22, 2012.
  22. Moscatelli, M., F. Parlapiano, S. Narizzano, and G. Viggiano, "Corporate default forecasting with machine learning", Expert Systems with Applications, Vol.161, No.113567, 2020, pp. 1-12.
  23. Murdoch, W. J., C. Singh, K. Kumbier, R. Abbasi-Asl, and B. Yu, "Definitions, methods, and applications in interpretable machine learning", Proceedings of the National Academy of Sciences of the United States of America, Vol.116, 2019, pp. 22071-80.
  24. Novak, P. K., N. Lavrac, and G. I. Webb, "Supervised descriptive rule discovery: A unifying survey of contrast set, emerging pattern and subgroup mining", Journal of Machine Learning Research, Vol.10, 2009, pp. 377-403.
  25. Poyiadzi, R., K. Sokol, R. Santos-rodriguez, T. De Bie, and P. Flach, "FACE: Feasible and actionable counterfactual explanations", AAAI/ACM Conference on AI, Ethics, and Society (AIES), 2020.
  26. Rajapaksha, D., C. Bergmeir, and W. Buntine, "LoRMIkA: Local Rule-Based Model Interpretability with k-Optimal Associations", Information Sciences, Vol.540, 2020, pp. 221-41.
  27. Ribeiro, M. T. and C. Guestrin, "'Why should I trust you?' Explaining the predictions of any classifier", KDD 2016, 2016.
  28. Ribeiro, M. T., S. Singh, and C. Guestrin, "Anchors: High-Precision model-agnostic explanations", The Thirty-Second AAAI Conference on Artificial Intelligence (AAAI-18), 2018, pp. 1527-35.
  29. Roscher, R., B. Bohn, M. F. Duarte, and J. Garcke, "Explainable machine learning for scientific insights and discoveries", IEEE Access, Vol.8, 2020, pp. 42200-216.
  30. Setiono, R. and H. Liu, "Symbolic representation of neural networks", Computer, 1996, pp. 71-77.
  31. Sharma, P., A. Wadhwa, and K. Komal, "Analysis of selection schemes for solving an optimization problem in genetic algorithm", International Journal of Computer Applications, Vol.93, No.11, 2014, pp. 1-3.
  32. Soui, M., I. Gasmi, S. Smiti, and K. Ghedira, "Rule-base d credit risk assessment model using multi-objective evolutionary algorithms", Expert Systems With Applications, Vol.126, 2019, pp. 144-57.
  33. Yi, J., "Credit scoring model based on the decision tree and the simulated annealing algorithm", 2009 WRI World Congress on Computer Science and Information Engineering, 2009, pp. 18-22.