Asia pacific journal of information systems

The Korea Society of Management Information Systems (한국경영정보학회)
권호 표지
DOI QR코드

DOI QR Code

Aspect-based Sentiment Analysis of Product Reviews using Multi-agent Deep Reinforcement Learning

  • M. Sivakumar (Epartment of computer science and engineering, K. Ramakrishnan College of Technology) ;
  • Srinivasulu Reddy Uyyala (National Institute of Technology (NIT))
  • Received : 2021.08.26
  • Accepted : 2022.03.04
  • Published : 2022.06.30
Download PDF
⟨ Previous Next ⟩

Abstract

The existing model for sentiment analysis of product reviews learned from past data and new data was labeled based on training. But new data was never used by the existing system for making a decision. The proposed Aspect-based multi-agent Deep Reinforcement learning Sentiment Analysis (ADRSA) model learned from its very first data without the help of any training dataset and labeled a sentence with aspect category and sentiment polarity. It keeps on learning from the new data and updates its knowledge for improving its intelligence. The decision of the proposed system changed over time based on the new data. So, the accuracy of the sentiment analysis using deep reinforcement learning was improved over supervised learning and unsupervised learning methods. Hence, the sentiments of premium customers on a particular site can be explored to other customers effectively. A dynamic environment with a strong knowledge base can help the system to remember the sentences and usage State Action Reward State Action (SARSA) algorithm with Bidirectional Encoder Representations from Transformers (BERT) model improved the performance of the proposed system in terms of accuracy when compared to the state of art methods.

Keywords

References

  1. Arulkumaran, K., Deisenroth, M. P., Brundage, M., and Bharath, A. A. (2017). Deep reinforcement learning: A brief survey. IEEE Signal Processing Magazine, 34(6), 26-38. https://doi.org/10.1109/MSP.2017.2743240
  2. Baccianella, S., Esuli, A., and Sebastiani, F. (2010). SentiWordNet 3.0: An enhanced lexical resource for sentiment analysis and opinion mining. Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC'10), 2200-2204.
  3. Boute, R. N., Gijsbrechts, J., van Jaarsveld, W., and Vanvuchelen, N. (2021). Deep reinforcement learning for inventory control: A roadmap. European Journal of Operational Research, 298(2), 401-412. https://doi.org/10.1016/j.ejor.2021.07.016.
  4. Chai, D., Wu, W., Han, Q., Wu, F., and Li, J. (2020). Description based text classification with reinforcement learning. International Conference on Machine Learning, 1371-1382.
  5. Chen, R., Zhou, Y., Zhang, L., and Duan, X. (2019). Word-level sentiment analysis with reinforcement learning. IOP Conference Series: Materials Science and Engineering, 490, 062063. https://doi.org/10.1088/1757-899X/490/6/062063
  6. Chu, T., Wang, J., Codeca, L., and Li, Z. (2020). Multi-agent deep reinforcement learning for large-scale traffic signal control. IEEE Transactions on Intelligent Transportation Systems, 21, 1086-1095. https://doi.org/10.1109/TITS.2019.2901791
  7. Cote, M., Kadar, A., Yuan, X., Kybartas, B. A., Barnes, T., Fine, E., Moore, J., Hausknecht, M. J., Asri, L. E., Adada, M., Tay, W., and Trischler, A. (2018). TextWorld a learning environment for text-based games. Computer Games Workshop at ICML/IJCAI. arXiv:1806.11532.
  8. Deshpande, A., and Fleisig, E. (2020). Sentiment Analysis for Reinforcement Learning. arXiv:2010.02316.
  9. Devlin, J., Chang, M., Lee, K., and Toutanova, K. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), 4171-4186.
  10. Diallo, E. A. O., Sugiyama, A., and Sugawara, T. (2020). Coordinated behavior of cooperative agents using deep reinforcement learning. Neuro computing, 396, 230-240.
  11. Elavarasan, D., and Vincent, P. M. (2020). Crop yield prediction using deep reinforcement learning model for sustainable agrarian applications. IEEE Access, 8, 86886-86901.
  12. Goh, O. S., Fung, C. C., and Depickere, A. (2008). Domain knowledge query conversation bots in instant messaging (IM). Knowledge-Based Systems, 21(7), 681-691. https://doi.org/10.1016/j.knosys.2008.03.055
  13. Goldwaser, A., and Thielscher, M. (2020). Deep reinforcement learning for general game playing. Proceedings of the AAAI Conference on Artificial Intelligence, 34(2), 1701-1708.
  14. Hasselt, H. V., Guez, A., and Silver, D. (2016). Deep reinforcement learning with double Q-learning. Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence (AAAI-16), 2094-2100.
  15. Jonsson, A. (2019), Deep reinforcement learning in medicine. Kidney Dis, 5, 18-22. https://doi.org/10.1159/000492670
  16. Keller, T. R., and Klinger, U. (2019). Social bots in election campaigns: Theoretical, empirical, and methodological implications. Political Communication, 36(1), 171-189. https://doi.org/10.1080/10584609.2018.1526238
  17. Koratamaddi, P., Wadhwani, K., Gupta, M., and Sanjeevi, S. G. (2021). Market sentiment-aware deep reinforcement learning approach for stock portfolio allocation. Engineering Science and Technology, an International Journal, 24(4), 848-859.
  18. Kulkarni, C. S., Bhavsar, A. U., Pingale, S. R., and Kumbhar, S. S. (2017). BANK CHAT BOT-an intelligent assistant system using NLP and machine learning. International Research Journal of Engineering and Technology, 4(5), 2374-2377.
  19. Liu, R., Nageotte, F., Zanne, P., Mathelin, M. D., and Dresp, B. (2021). Deep reinforcement learning for the control of robotic manipulation: A focussed mini-review. Robotics, 10(22), 1-13.
  20. Meenakshi, Banerjee, A., Intwala, N., and Sawant, V. (2020). Sentiment analysis of amazon mobile reviews. In M. Tuba, S. Akashe, and A. Joshi (Eds.), ICT Systems and Sustainability. Advances in Intelligent Systems and Computing, 1077. Singapore. https://doi.org/10.1007/978-981-15-0936-0_4
  21. McAuley, J., and Leskovec, J. (2013). From amateurs to connoisseurs: modeling the evolution of user expertise through online reviews. Proceedings of the 22nd international conference on World Wide Web.
  22. Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., and Riedmiller, M. (2013). Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602.
  23. Pang, B., and Lee, L. (2005). Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales. Association for Computational Linguistics, 115-124.
  24. Peng, H., Ma, Y., Poria, S., Li, Y., and Cambria, E. (2021). Phonetic-enriched text representation for chinese sentiment analysis with reinforcement learning. Information Fusion,70, 88-99. https://doi.org/10.1016/j.inffus.2021.01.005
  25. Prollochs, N., Feuerriegel, S., Lutz, B., and Neumann, D. (2020). Negation scope detection for sentiment analysis a reinforcement learning framework for replicating human interpretations. Information Sciences, 536, 205-221.
  26. Rowley, J. (2000). Product searching with shopping bots. Internet Research, 10(3), 203-214. https://doi.org/10.1108/10662240010331957
  27. Shahmanzari, M., and Ozkan, S. (2014). Accessing the effect of e-commerce intelligent bots on online consumers' post-adoption behavior for future use. American Academic & Scholarly Research Journal, 6(4), 163.
  28. Wang, J., Sun, C., Li, S., Wang, J., Si, L., Zhang, M., Liu, X., and Zhou, G. (2019). Human-like decision making: Document-level aspect sentiment classification via hierarchical reinforcement learning. EMNLP, 5581-5590.
  29. Wang, W. Y., Li, J., and He, X. (2018). Deep reinforcement learning for NLP. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics: Tutorial Abstracts, 19-21.
  30. Yang, M., Jiang, Q., Shen, Y., Wu, Q., Zhao, Z., and Zhou, W. (2019). Hierarchical human-like strategy for aspect-level sentiment classification with sentiment linguistic knowledge and reinforcement learning. Neural Networks: The Official Journal of the International Neural Network Society, 117, 240-248. https://doi.org/10.1016/j.neunet.2019.05.021
  31. Yu, L., Gao, Z., Qin, S., Zhang, M., Shen, C., Guan, X., and Yue, D. (2020). Deep Reinforcement Learning for Smart Grid Protection Against Coordinated Multistage Transmission Line Attacks. arXiv:2011. 14526.
  32. Yu, S., Nayak, T., Majumder, N., and Poria, S. (2021). Aspect sentiment triplet extraction using reinforcement learning. Proceedings of the 30th ACM International Conference on Information & Knowledge Management, 3603-3607.
  33. Zhang, C. (2019). Sentiment Analysis and Deep Reinforcement Learning for Algorithmic Trading. Retrieved from http://ink-ron.usc.edu/xiangren/ml4know19spring/public/midterm/Chi_Zhang_Report.pdf
  34. Zhang, D., Li, S., Zhu, Q., and Zhou, G. (2019). Modeling the clause-level structure to multimodal sentiment analysis via reinforcement learning. 2019 IEEE International Conference on Multimedia and Expo (ICME), 730-735.
  35. Zhao, D., Wang, H., Shao, K., and Zhu, Y. (2016). Deep reinforcement learning with experience replay based on SARSA. 2016 IEEE Symposium Series on Computational Intelligence (SSCI), 1-6.