Browse > Article
http://dx.doi.org/10.3745/KTCCS.2019.8.1.17

Blockchain Based Financial Portfolio Management Using A3C  

Kim, Ju-Bong (한국기술교육대학교 컴퓨터공학부)
Heo, Joo-Seong (한국기술교육대학교 컴퓨터공학부)
Lim, Hyun-Kyo (한국기술교육대학교 창의융합공학협동과정 ICT융합)
Kwon, Do-Hyung (한국기술교육대학교 창의융합공학협동과정 ICT융합)
Han, Youn-Hee (한국기술교육대학교 컴퓨터공학부)
Publication Information
KIPS Transactions on Computer and Communication Systems / v.8, no.1, 2019 , pp. 17-28 More about this Journal
Abstract
In the financial investment management strategy, the distributed investment selecting and combining various financial assets is called portfolio management theory. In recent years, the blockchain based financial assets, such as cryptocurrencies, have been traded on several well-known exchanges, and an efficient portfolio management approach is required in order for investors to steadily raise their return on investment in cryptocurrencies. On the other hand, deep learning has shown remarkable results in various fields, and research on application of deep reinforcement learning algorithm to portfolio management has begun. In this paper, we propose an efficient financial portfolio investment management method based on Asynchronous Advantage Actor-Critic (A3C), which is a representative asynchronous reinforcement learning algorithm. In addition, since the conventional cross-entropy function can not be applied to portfolio management, we propose a proper method where the existing cross-entropy is modified to fit the portfolio investment method. Finally, we compare the proposed A3C model with the existing reinforcement learning based cryptography portfolio investment algorithm, and prove that the performance of the proposed A3C model is better than the existing one.
Keywords
Reinforcement Learning; Financial Portfolio Management; A3C; Cryptocurrency; Investment Engineering;
Citations & Related Records
연도 인용수 순위
  • Reference
1 Nakamoto, Satoshi, Bitcoin: A Peer-to-Peer Electronic Cash System, Cryptography Mailing list at https://metzdowd.com, 2009.
2 "GUNBOT - Crypto Trading Bot," GUNBOT, https://www.gunbot.com, 2018.
3 "start [ProfitTrailer Wiki]", ProfitTrailer, https://wiki.profittrailer.com/doku.php?id=start, 2018.
4 I. Kaastra and M. Boyd, “Designing a neural network for forecasting financial and economic time series,” Neurocomputing, Vol. 10, No. 3, pp. 215-236, 1996.   DOI
5 Candela, "Dataset shift in machine learning," London: MIT Press, 006.3 CAN, 2009.
6 Y. B, Kim, "Predicting Fluctuations in Cryptocurrency Transactions Based on User Comments and Replies," PLoS ONE, Vol. 11, No. 8, e0161197, 2016.   DOI
7 Sean McNally, "Predicting the Price of Bitcoin Using Machine Learning," 26th Euromicro International Conference on Parallel, Distributed and Network-based Processing, pp. 339-343, Mar. 2018.
8 R. Sutton and A. Barto, "Reinforcement Learning: an Introduction," MIT Press, 1998.
9 Volodymyr Mnih, "Asynchronous Methods for Deep Reinforcement Learning," Proceedings of the 33rd International Conference on MachineLearning, New York, NY, USA, 2016. JMLR: W&CP volume48.
10 Arun Nair, "Massively Parallel Methods for Deep Reinforcement Learning," at Deep Learning Workshop, International Conference on Machine Learning, Lille, France, 2015.
11 Zhengyao Jiang, "A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problem," In JMLR, 30 pages, 5 figures, 2017.
12 Christopher JCH Watkins and Peter Dayan. "Q-Learning," Machine Learning, Vol. 8, No. 3-4, pp. 279-292, 1992.   DOI
13 Kai Arulkumaran, "A Brief Survey of Deep Reinforcement Learning," in IEEE Signal Processing Magazine Special Issue On Deep Learning For Image Understanding, 2017.
14 Hado van Hasselt, "Deep Reinforcement Learning with Double Q-learning," Proceedings of 30th AAAI Conference on Artificial Intelligence (AAAI-16).
15 Gaivoronski, "Stochastic nonstationary optimization for finding universal portfolios," in Annals of Operations Research, Vol. 100, No. 1, pp. 165-188, 2000.   DOI
16 David Silver, Guy Lever, Nicolas Heess, Thomas Degris, Daan Wierstra, and Martin Ried-miller, "Deterministic Policy Gradient Algorithms," ICML(International Conference on Machine Learning) Proceedings of the 31st, pp. 387-395, 2014.
17 K. Chepuri, T. Homem de Mello, "Solving the vehicle routing problem with stochastic demands using the cross entropy method," Annals of Operations Research, 2004.
18 G. Alon, D. P. Kroese, T. Raviv, and R. Y. Rubinstein, "Application of the Cross-entropy method to the buffer allocation problem in a simulation-based environment," Annals of Operations Research, 2004.
19 Agarwal, A., "Algorithms for portfolio management basedon the newton method," in ICML, New York, NY, USA (2006)
20 Bin Li, Peilin Zhao, Steven C. H. Hoi, and Vivekanand Gopalkrishnan. "Passive aggressive mean reversion strategy for portfolio selection," PSMR, Machine Learning, Vol. 87, No. 2, pp. 221-258, 2012.   DOI
21 Seyed Taghi Akhavan Niaki and Saeid Hoseinzade. "Forecasting S&P 500 index using artificial neural networks and design of experiments," Journal of Industrial Engineering International, Vol. 9, No. 1, p.1, 2013.   DOI
22 Katia Sycara, K. Decker and Dajun Zeng, "Designing a Multi-Agent Portfolio Management System," Proceedings of the AAAI Workshop on Internet Information Systems, 1995.
23 K. Sycara, A. Pannu, M. Willamson, Dajun Zeng, K. Decker, "Distributed intelligent agents," IEEE Expert, Vol. 11, Issue 6, Dec. 1996.
24 Mu Li, "Efficient Mini-batch Training for Stochastic Optimization," In 2014 ACM, 978-1-4503-2956-9, 2014.
25 Hiroshi Takahashi, "Analyzing the Effectiveness of Investment Strategies through Agent-based Modelling: Overconfident Investment Decision Making and Passive Investment Strategies," eKNOW, 6th International Conference, 2014.
26 "API - Bithumb," Bithumb, https://www.bithumb.com/u1/US127, 2018.
27 Mnih, Volodymyr, “Human-level control through deep reinforcementlearning,” Nature, Vol. 518, No. 7540, pp. 529-533, 2015.   DOI
28 Edward Qian, “Active Risk And Information Ratio,” Journal of Investment Management, Vol. 2, No. 3, pp. 1-15, 2004.   DOI