Journal of the Architectural Institute of Korea (대한건축학회논문집)

Architectural Institute of Korea (대한건축학회)
권호 표지
DOI QR코드

DOI QR Code

Epistemic and Aleatoric Uncertainty of Bayesian Neural Network Model for a Chiller

베이지안 신경망 냉동기 모델의 인식론적 및 내재적 불확실성 분석

  • Kim, Jae Min ;
  • Park, Cheol Soo (Department of Architecture and Architectural Engineering.Institute of Engineering Research, Seoul National University)
  • Received : 2020.04.24
  • Accepted : 2020.06.01
  • Published : 2020.06.30
⟨ Previous Next ⟩

Abstract

Because the machine learning model is a black-box model, it is difficult to quantify the causality between inputs and outputs. In addition, the model is influenced by its inherent uncertainty in describing a system's behavior of interest. In order for a machine learning model to be reliable, its prediction performance as well as uncertainty must be quantified together. Bayesian Neural Network (BNN) is a useful tool to describe stochastic characteristics of deep learning models by estimating distributions of the models' weights. Model uncertainty in BNN can be classified into epistemic and aleatoric uncertainties. Epistemic uncertainty is caused by lack of data or knowledge. In contrast, aleatoric uncertainty is caused by outliers or noises inherent in training data and can be reduced by removing abnormal data from the training dataset. In this study, the BNN models were developed for a compression chiller in an existing office building with BEMS data, and then epistemic and aleatoric uncertainties were analyzed. It is found that both uncertainties are significant in the simulation model even though the model's accuracy is satisfactory with the CVRMSE of less than 15%. It is suggested that before attempting to apply the machine learning model to real applications, the both uncertainties must be carefully analyzed. It is recommended that the both uncertainties can be reduced by adding more data as well as removing outliers.

Keywords

Acknowledgement

이 연구는 산업통상자원부(MOTIE)와 한국에너지기술평가원(KETEP)의 지원을 받아 수행한 연구 과제입니다. (No. 20182010106460)

References

  1. Afram, A., & Janabi-Sharifi, F. (2014). Review of modeling methods for HVAC systems. Applied Thermal Engineering. 67, pp.507-519. https://doi.org/10.1016/j.applthermaleng.2014.03.055
  2. Afram, A., Janabi-Sharifi, F., Fung, A. S., & Raahemifar, K. (2017). Artificial neural network based model predictive control and optimization of HVAC systems: A state of the art review and case study of a residential HVAC system. Energy and Buildings. 141, pp.96-113. https://doi.org/10.1016/j.enbuild.2017.02.012
  3. ASHRAE (2014). ASHRAE Guideline 14-2014: measurement of energy and demand savings. American Society of Heating, Refrigerating and Air-conditioning Engineers, Atlanta, GA
  4. Barber, D., & Bishop, C. M. (1998). Ensemble learning in Bayesian neural networks. Nato ASI Series F Computer and Systems Sciences, 168, (pp. 215-238).
  5. Blundell, C., Cornebise, J., Kavukcuoglu, K., & Wierstra, D. (2015). Weight uncertainty in neural networks. arXiv preprint arXiv:1505.05424.
  6. Der Kiureghian, A., & Ditlevsen, O. (2009). Aleatory or epistemic? Does it matter?. Structural safety, 31(2), 105-112. https://doi.org/10.1016/j.strusafe.2008.06.020
  7. Every, P. M. V., Rodriguez, M., Jones, C. B., Mammoli, A. A., & Martinez-Ramon, M. (2017). Adavanced detection of HVAC faults using unsupervised SVM novelty detection and Gaussian process models. Energy and Buildings. 149, pp.216-224. https://doi.org/10.1016/j.enbuild.2017.05.053
  8. Gal, Y., & Ghahramani, Z. (2016). Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In international conference on machine learning (pp. 1050-1059).
  9. Gal, Y. (2016). Uncertainty in deep learning. PhD Thesis. University of Cambridge.
  10. Hinton, G. E., & Van Camp, D. (1993). Keeping the neural networks simple by minimizing the description length of the weights. In Proceedings of the sixth annual conference on Computational learning theory (pp. 5-13).
  11. Hinton, G. E., Srivastava, N., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. R. (2012). Improving neural networks by preventing co-adaptation of feature detectors. arXiv preprint arXiv:1207.0580.
  12. Kendall, A. G., & Gal, Y. (2017). What uncertainties do we need in bayesian deep learning for computer vision?. In Advances in neural information processing systems (pp. 5574-5584).
  13. Kendall, A. G. (2019). Geometry and uncertainty in deep learning for computer vision. PhD Thesis. University of Cambridge.
  14. Kim, H., Jung, D. C., & Choi, B. W. (2019). Exploiting the Vulnerability of Deep Learning-Based Artificial Intelligence Models in Medical Imaging: Adversarial Attacks. Journal of the Korean Society of Radiology, 80 (2):259 https://doi.org/10.3348/jksr.2019.80.2.259
  15. Kwon, Y., Won, J. H., Kim, B. J., & Paik, M. C. (2018). Uncertainty quantification using bayesian neural networks in classification: Application to ischemic stroke lesion segmentation.
  16. Kwon, Y., Won, J. H., Kim, B. J., & Paik, M. C. (2020). Uncertainty quantification using bayesian neural networks in classification: Application to biomedical image segmentation. Computational Statistics & Data Analysis, 142, 106816. https://doi.org/10.1016/j.csda.2019.106816
  17. Neal, R. M. (1995). Bayesian learning for neural networks. PhD thesis, University of Toronto.
  18. Nikolaidou, E., Wright, J., & Hopfe, C. J. (2015, December). Early and detailed design stage modelling using Passivhaus design; what is the difference in predicted building performance. In BS2015, 14th Conference of International Building Performance Simulation Association, Hyderabad, India, December 7 (Vol. 9, pp. 2166-2173).
  19. Park, S. H., Ahn, K. U., Hwang, S. H., Choi, S. K., & Park, C. S. (2019). Machine learning vs. hybrid machine learning model for optimal operation of a chiller. Science and Technology for the Built Environment. 25, pp.209-220. https://doi.org/10.1080/23744731.2018.1510270
  20. Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. (2014). Dropout: a simple way to prevent neural networks from overfitting. The journal of machine learning research, 15(1), 1929-1958.
  21. Tax, D. M. J., & Duin, R. P. W. (2004). Support vector data description. Machine Learning, 54(1), (pp. 45-66). https://doi.org/10.1023/B:MACH.0000008084.60811.49
  22. Williams, C. K. (1997). Computing with infinite networks. NIPS.
  23. Williams, C. K., & Rasmussen, C. E. (2006). Gaussian processes for machine learning (Vol. 2). Cambridge, MA: MIT press.