Browse > Article
http://dx.doi.org/10.33851/JMIS.2020.7.1.1

A Comparison of Deep Reinforcement Learning and Deep learning for Complex Image Analysis  

Khajuria, Rishi (Department of CS&IT University of Jammu)
Quyoom, Abdul (Department of CS&IT University of Jammu)
Sarwar, Abid (Department of CS&IT University of Jammu)
Publication Information
Journal of Multimedia Information System / v.7, no.1, 2020 , pp. 1-10 More about this Journal
Abstract
The image analysis is an important and predominant task for classifying the different parts of the image. The analysis of complex image analysis like histopathological define a crucial factor in oncology due to its ability to help pathologists for interpretation of images and therefore various feature extraction techniques have been evolved from time to time for such analysis. Although deep reinforcement learning is a new and emerging technique but very less effort has been made to compare the deep learning and deep reinforcement learning for image analysis. The paper highlights how both techniques differ in feature extraction from complex images and discusses the potential pros and cons. The use of Convolution Neural Network (CNN) in image segmentation, detection and diagnosis of tumour, feature extraction is important but there are several challenges that need to be overcome before Deep Learning can be applied to digital pathology. The one being is the availability of sufficient training examples for medical image datasets, feature extraction from whole area of the image, ground truth localized annotations, adversarial effects of input representations and extremely large size of the digital pathological slides (in gigabytes).Even though formulating Histopathological Image Analysis (HIA) as Multi Instance Learning (MIL) problem is a remarkable step where histopathological image is divided into high resolution patches to make predictions for the patch and then combining them for overall slide predictions but it suffers from loss of contextual and spatial information. In such cases the deep reinforcement learning techniques can be used to learn feature from the limited data without losing contextual and spatial information.
Keywords
Deep reinforcement learning; Deep learning; Complex images; CNN; DQN;
Citations & Related Records
연도 인용수 순위
  • Reference
1 Hamidinekoo Azam et al., "Deep learning in mammography and breast histology, an overview and future trends," Medical image analysis, vol. 47, pp. 45-67, 2018.   DOI
2 Zhao Dongbin et al., "Deep reinforcement learning with experience replay based on SARSA," in Proceedings of IEEE Symposium Series on Computational Intelligence (SSCI), pp 1-6, 2016.
3 Ji-Hae Kim, Gwang-Soo Hong, Byung-Gyu Kim, Debi P. Dogra, "deepGesture: Deep Learning-based Gesture Recognition Scheme using Motion Sensors," Displays, vol. 55, pp. 38-45, 2018.   DOI
4 Ji-Hae Kim, Byung-Gyu Kim, Partha Pratim Roy, Da-Mi Jeong, "Efficient Facial Expression Recognition Algorithm Based on Hierarchical Deep Neural Network Structure," IEEE Access, vol. 7, pp. 41273-41285, 2019.   DOI
5 Mnih, Volodymyr, Nicolas Heess, and Alex Graves, "Recurrent models of visual attention," Advances in neural information processing systems, pp 2204-2212, 2014.
6 Bellman, Richard. "A Markovian decision process." Journal of mathematics and mechanics, pp. 679-684, 1957.
7 Konda, Vijay R., and John N. Tsitsiklis, "Onactor-critic algorithms," SIAM journal on Control and Optimization, vol. 42, no. 4, pp. 1143-1166, 2003.   DOI
8 Beard, Randal W., George N. Saridis, and John T. Wen. "Galerkin approximations of the generalized Hamilton-Jacobi-Bellman equation," Automatica, vol. 33, no. 12, pp. 2159-2177, 1997.   DOI
9 Busoniu, Lucian et al., Reinforcement learning and dynamic programming using function approximators, CRC press, 2017.
10 Precup, Doina, Richard S. Sutton, and Sanjoy Dasgupta. "Off-policy temporal-difference learning with function approximation," in Proceedings of ICML, pp. 417-424, 2001.
11 Saunders, William, et al. "Trial without error: Towards safe reinforcement learning via human intervention," in Proceedings of the 17th International Conference on Autonomous Agents and MultiAgent Systems. International Foundation for Autonomous Agents and Multiagent Systems, pp. 2067-2069, 2018.
12 LeCun, Yann, Yoshua Bengio, and Geoffrey Hinton, "Deep learning," Nature 521.7553, pp. 436-444, 2015.   DOI
13 Bottou, Leon. "Stochastic gradient learning in neural networks," in Proceedings of Neuro-Nimes 91.8, vol. 12, 1991.
14 Kaelbling, Leslie Pack, Michael L. Littman, and Andrew W. Moore. "Reinforcement learning: A survey," Journal of artificial intelligence research, vol. 4, pp. 237-285, 1996.   DOI
15 Benvenuto, Nevio, and Francesco Piazza, "On the complex backpropagation algorithm," IEEE Transactions on Signal Processing, vol. 40, no. 4, pp. 967-969, 1992.   DOI
16 Price, Micah, et al., "Object detection using image classification models," U.S. Patent No. 10,223,611. 5 Mar. 2019.
17 Graves, Alex, and Navdeep Jaitly, "Towards end-to-end speech recognition with recurrent neural networks," in Proceedings of International conference on machine learning, pp. 1764-1772, 2014.
18 Cho Kyunghyun et al., "Learning phrase representations using RNN encoder-decoder for statistical machine translation," arXiv preprint arXiv:1406.1078, 2014.
19 McCoppin, Ryan, and Mateen Rizki, "Deep learning for image classification," Ground/Air Multisensor Interoperability, Integration, and Networking for Persistent ISR V, vol. 9079. International Society for Optics and Photonics, pp. 946401-1, 2014.
20 Krizhevsky, Alex, Ilya Sutskever, and Geoffrey E. Hinton, "Imagenet classification with deep convolutional neural networks," Advances in neural information processing systems, pp. 1097-1105, 2012.
21 You, Quanzeng et al., "Image captioning with semantic attention," in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4651-4659, 2016.
22 Naylor Peter et al., "Nuclei segmentation in histopathology images using deep neural networks," in Proceedings of IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017), pp. 933-936, 2017.
23 Ullah, Amin, et al., "Action recognition in video sequences using deep bi-directional LSTM with CNN features," IEEE Access, vol. 6, pp. 1155-1166, 2017.   DOI
24 Wang, Naiyan, and Dit-Yan Yeung, "Learning a deep compact image representation for visual tracking," Advances in neural information processing systems, pp. 809-817, 2013.
25 Mnih, Volodymyr et al., "Human-level control through deep reinforcement learning," Nature 518.7 540-529, 2015.
26 Le, Quoc V., "Building high-level features using large scale unsupervised learning," in Proceedings of the IEEE international conference on acoustics, speech and signal processing, pp. 8595-8598, 2013.
27 Radford, Alec, Luke Metz, and Soumith Chintala, "Unsupervised representation learning with deep convolutional generative adversarial networks," arXiv preprint arXiv:1511.06434, 2015.
28 Mnih, Volodymyr et al., "Playing atari with deep reinforcement learning," arXiv preprint arXiv:1312.5602, 2013.
29 Watkins, Christopher JCH, and Peter Dayan, "Q-learning." Machine learning, vol. 8, no. 3-4, pp. 279-292, 1992.   DOI
30 Rust, John, "Using randomization to break the curse of dimensionality," Econometrica: Journal of the Econometric Society, pp. 487-516, 1997.
31 Nachum, Ofir, et al., "Bridging the gap between value and policy based reinforcement learning," Advances in Neural Information Processing Systems, pp. 2775-2785, 2017.
32 Caicedo, Juan C., and Svetlana Lazebnik, "Active object localization with deep reinforcement learning," in Proceedings of the IEEE International Conference on Computer Vision, pp. 2488-2496, 2015.
33 Grudic, Gregory Z., Vijay Kumar, and Lyle Ungar, "Using policy gradient reinforcement learning on autonomous robot controllers," in Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003) (Cat. No. 03CH37453). vol. 1, pp. 406-411, 2003.
34 Mikolov, Tomas, et al., "Recurrent neural network based language model," in Proceedings of Eleventh annual conference of the international speech communication association, 2010
35 Nixon, Mark, and Alberto Aguado, Feature extraction and image processing for computer vision. Academic press, 2019
36 Bakator, Mihalj, and Dragica Radosav, "Deep learning and medical diagnosis: A review of literature," Multimodal Technologies and Interaction, vol. 2, no. 3 2018.
37 Maicas, Gabriel, et al., "Deep reinforcement learning for active breast lesion detection from DCE-MRI," in Proceedings of International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer, Cham, pp. 665-673, 2017.
38 Ghesu, Florin C. et al., "An artificial agent for anatomical landmark detection in medical images," in Proceedings of International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer, Cham, pp. 229-237, 2016.
39 Momeni, Alexandre, Marc Thibault, and Olivier Gevaert, "Deep Recurrent Attention Models for Histopathological Image Analysis," BioRxiv: 438341, 2018.
40 Mnih, Volodymyr, Nicolas Heess, and Alex Graves, "Recurrent models of visual attention," Advances in neural information processing systems, pp. 2204-2212, 2014.
41 Ali, Issa et al., "Lung nodule detection via deep reinforcement learning," Frontiers in oncology, vol. 8, 2008.
42 Raghu, Aniruddh et al., "Deep reinforcement learning for sepsis treatment," arXiv preprint arXiv:1711.09602, 2017.
43 Martinez Coralie et al., "A deep reinforcement learning approach for early classification of time series," in Proceedings of the 26th European Signal Processing Conference (EUSIPCO), pp. 2030-2034, 2018.
44 Volodymyr Mnih et al., "Human-level control through deep reinforcement learning," Nature 518.7540, pp. 529-533, 2015.   DOI
45 Chen Yanping et al., "The ucr time series classification archive," http://www.cs.ucr.edu/-eamonn/time_series_data, 2015.
46 Dinis Adriana, Todor Ivascu, and Viorel Negru, "A Self Developing System for Medical Data Analysis," in Proceedings of the 20th International Symposium on Symbolic and Numeric Algorithms for Scientific Computing (SYNASC), pp. 335-339, 2018.
47 Wang Zi et al., "Deep reinforcement learning of cell movement in the early stage of C. elegans embryogenesis," Bioinformatics, vol. 34, no. 18, pp. 3169-3177, 2018.   DOI
48 Liu Ying et al., "Deep reinforcement learning for dynamic treatment regimens on medical registry data," in Proceedings of IEEE International Conference on Healthcare Informatics (ICHI), pp. 380-385, 2017.
49 Rakhlin Alexander et al., "Deep convolutional neural networks for breast cancer histology image analysis," in Proceedings of International Conference Image Analysis and Recognition, Springer, Cham, pp. 737-744, 2018.
50 Turkki Riku et al., "Breast cancer outcome prediction with tumour tissue images and machine learning," Breast cancer research and treatment, pp. 1-12, 2019.
51 Bychkov Dmitrii et al., "Deep learning based tissue analysis predicts outcome in colorectal cancer," Scientific Reports, vol. 8, no.1: 3395, 2018.   DOI
52 Grewal Parampal S. et al., "Deep learning in ophthalmology: a review," Canadian Journal of Ophthalmology, vol. 53, no. 4, pp. 309-313, 2018.   DOI