한국정보처리학회:학술대회논문집 (Proceedings of the Korea Information Processing Society Conference)
- 한국정보처리학회 2023년도 춘계학술발표대회
- /
- Pages.624-626
- /
- 2023
- /
- 2005-0011(pISSN)
- /
- 2671-7298(eISSN)
DOI QR Code
An Ensemble Model for Credit Default Discrimination: Incorporating BERT-based NLP and Transformer
- Sophot Ky (Dept. of Electrical and Computer Engineering, Inha University) ;
- Ju-Hong Lee (Dept. of Computer Engineering, Inha University)
- 발행 : 2023.05.18
초록
Credit scoring is a technique used by financial institutions to assess the creditworthiness of potential borrowers. This involves evaluating a borrower's credit history to predict the likelihood of defaulting on a loan. This paper presents an ensemble of two Transformer based models within a framework for discriminating the default risk of loan applications in the field of credit scoring. The first model is FinBERT, a pretrained NLP model to analyze sentiment of financial text. The second model is FT-Transformer, a simple adaptation of the Transformer architecture for the tabular domain. Both models are trained on the same underlying data set, with the only difference being the representation of the data. This multi-modal approach allows us to leverage the unique capabilities of each model and potentially uncover insights that may not be apparent when using a single model alone. We compare our model with two famous ensemble-based models, Random Forest and Extreme Gradient Boosting.
키워드