Progress in Medical Physics (한국의학물리학회지:의학물리)

Korean Society of Medical Physics (한국의학물리학회)
권호 표지
DOI QR코드

DOI QR Code

Deep Learning in Radiation Oncology

  • Cheon, Wonjoong (Proton Therapy Center, National Cancer Center) ;
  • Kim, Haksoo (Proton Therapy Center, National Cancer Center) ;
  • Kim, Jinsung (Department of Radiation Oncology, Yonsei Cancer Center, Yonsei University College of Medicine)
  • Received : 2020.08.10
  • Accepted : 2020.09.03
  • Published : 2020.09.30
Download PDF
⟨ Previous Next ⟩

Abstract

Deep learning (DL) is a subset of machine learning and artificial intelligence that has a deep neural network with a structure similar to the human neural system and has been trained using big data. DL narrows the gap between data acquisition and meaningful interpretation without explicit programming. It has so far outperformed most classification and regression methods and can automatically learn data representations for specific tasks. The application areas of DL in radiation oncology include classification, semantic segmentation, object detection, image translation and generation, and image captioning. This article tries to understand what is the potential role of DL and what can be more achieved by utilizing it in radiation oncology. With the advances in DL, various studies contributing to the development of radiation oncology were investigated comprehensively. In this article, the radiation treatment process was divided into six consecutive stages as follows: patient assessment, simulation, target and organs-at-risk segmentation, treatment planning, quality assurance, and beam delivery in terms of workflow. Studies using DL were classified and organized according to each radiation treatment process. State-of-the-art studies were identified, and the clinical utilities of those researches were examined. The DL model could provide faster and more accurate solutions to problems faced by oncologists. While the effect of a data-driven approach on improving the quality of care for cancer patients is evidently clear, implementing these methods will require cultural changes at both the professional and institutional levels. We believe this paper will serve as a guide for both clinicians and medical physicists on issues that need to be addressed in time.

Keywords

Acknowledgement

Photographs courtesy of Sang Hee Ahn (National Cancer Center, Goyang), Jaehee Chun (Yonsei Cancer Center, Seoul), and Sang Woon Jeong (Samsung Medical Center, Seoul).

References

  1. Goodfellow I, Bengio Y, Courville A. Deep learning. Massachusetts: MIT Press; 2016.
  2. Bini SA. Artificial intelligence, machine learning, deep learning, and cognitive computing: what do these terms mean and how will they impact health care? J Arthroplasty. 2018;33:2358-2361. https://doi.org/10.1016/j.arth.2018.02.067
  3. Garrido A. Brain and artificial intelligence. Brain. 2017;8: 85-90.
  4. Turing AM. I.-Computing machinery and intelligence. Mind. 1950;59:433-460. https://doi.org/10.1093/mind/LIX.236.433
  5. McCulloch WS, Pitts W. A logical calculus of the ideas immanent in nervous activity. Bull Math Biol. 1943;5:15-133.
  6. McCorduck P. Machines who think. Natick: A K Peters, Ltd.; 2004.
  7. Rosenblatt F. Principles of neurodynamics: perceptrons and the theory of brain mechanisms. Washington D.C.: Spartan Books; 1961.
  8. Howe J. Artificial intelligence at Edinburgh University: a perspective. Edinburgh: the University of Edinburgh, 2007 [cited 2020 Jul 23]. Available from: http://www.inf.ed.ac.uk/about/AIhistory.html.
  9. Russell SJ, Norvig P. Artificial intelligence: a modern approach. 2nd ed. Upper Saddle River: Pearson Education, Inc.; 2003.
  10. Nair V, Hinton GE. Rectified linear units improve restricted Boltzmann machines. Paper presented at: ICML'10: Proceedings of the 27th International Conference on International Conference on Machine Learning; 2010 Jun 21-25; Haifa, Israel. p. 807-814.
  11. Krizhevsky A, Sutskever I, Hinton GE. ImageNet classification with deep convolutional neural networks. 2012:1097-1105.
  12. He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. Ithaca: arXiv.org, 2015 [cited 2020 Jul 23]. Available from: https://arxiv.org/abs/1512.03385.
  13. Goodfellow IJ, Pouget-Abadie J, Mirza M, Xu B, WardeFarley D, Ozair S, et al. Generative adversarial networks. Ithaca: arXiv.org, 2014 [cited 2020 Jul 23]. Available from: https://arxiv.org/abs/1406.2661.
  14. Hochreiter S, Schmidhuber J. Long short-term memory. Neural Comput. 1997;9:1735-1780. https://doi.org/10.1162/neco.1997.9.8.1735
  15. Gers FA, Schmidhuber J. Recurrent nets that time and count. Paper presented at: the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium; 2000 Jul 27; Como, Italy. p. 189-194.
  16. Cho K, van Merrienboer B, Gulcehre C, Bahdanau D, Bougares F, Schwenk H, et al. Learning phrase representations using RNN encoder-decoder for statistical machine translation. Ithaca: arXiv.org, 2014 [cited 2020 Jul 23]. Available from: https://arxiv.org/abs/1406.1078.
  17. Schuster M, Paliwal KK. Bidirectional recurrent neural networks. IEEE Trans Signal Process. 1997;45:2673-2681. https://doi.org/10.1109/78.650093
  18. Krause B, Lu L, Murray I, Renals S. Multiplicative LSTM for sequence modelling. Ithaca: arXiv.org, 2016 [cited 2020 Jul 23]. Available from: https://arxiv.org/abs/1609.07959.
  19. Wu Y, Schuster M, Chen Z, Le QV, Norouzi M, Macherey W, et al. Google's neural machine translation system: bridging the gap between human and machine translation. Ithaca: arXiv.org, 2016 [cited 2020 Jul 23]. Available from: https://arxiv.org/abs/1609.08144.
  20. Cui S, Tseng HH, Pakela J, Ten Haken RK, El Naqa I. Introduction to machine and deep learning for medical physicists. Med Phys. 2020;47:e127-e147.
  21. Hanley J, Debois MM, Mah D, Mageras GS, Raben A, Rosenzweig K, et al. Deep inspiration breath-hold technique for lung tumors: the potential value of target immobilization and reduced lung density in dose escalation. Int J Radiat Oncol Biol Phys. 1999;45:603-611. https://doi.org/10.1016/S0360-3016(99)00154-6
  22. Kubo HD, Hill BC. Respiration gated radiotherapy treatment: a technical study. Phys Med Biol. 1996;41:83-91. https://doi.org/10.1088/0031-9155/41/1/007
  23. Ramsey CR, Scaperoth D, Arwood D, Oliver AL. Clinical efficacy of respiratory gated conformal radiation therapy. Med Dosim. 1999;24:115-119. https://doi.org/10.1016/S0958-3947(99)00006-0
  24. Shirato H, Shimizu S, Kunieda T, Kitamura K, van Herk M, Kagei K, et al. Physical aspects of a real-time tumor-tracking system for gated radiotherapy. Int J Radiat Oncol Biol Phys. 2000;48:1187-1195. https://doi.org/10.1016/S0360-3016(00)00748-3
  25. de Kruijff RM. FLASH radiotherapy: ultra-high dose rates to spare healthy tissue. Int J Radiat Biol. 2020;96:419-423. https://doi.org/10.1080/09553002.2020.1704912
  26. Sun WZ, Jiang MY, Ren L, Dang J, You T, Yin FF. Respiratory signal prediction based on adaptive boosting and multilayer perceptron neural network. Phys Med Biol. 2017;62: 6822-6835. https://doi.org/10.1088/1361-6560/aa7cd4
  27. Wang R, Liang X, Zhu X, Xie Y. A feasibility of respiration prediction based on deep Bi-LSTM for real-time tumor tracking. IEEE Access. 2018;6:51262-51268. https://doi.org/10.1109/ACCESS.2018.2869780
  28. Chaikh A, Thariat J, Thureau S, Tessonnier T, Kammerer E, Fontbonne C, et al. [Construction of radiobiological models as TCP (tumor control probability) and NTCP (normal tissue complication probability): from dose to clinical effects prediction]. Cancer Radiother. 2020;24:247-257. French. https://doi.org/10.1016/j.canrad.2019.12.004
  29. Luo Y, Chen S, Valdes G. Machine learning for radiation outcome modeling and prediction. Med Phys. 2020;47:e178-e184.
  30. Arefan D, Mohamed AA, Berg WA, Zuley ML, Sumkin JH, Wu S. Deep learning modeling using normal mammograms for predicting breast cancer risk. Med Phys. 2020;47: 110-118. https://doi.org/10.1002/mp.13886
  31. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, et al. Going deeper with convolutions. Ithaca: arXiv.org, 2014 [cited 2020 Jul 23]. Available from: https://arxiv.org/abs/1409.4842.
  32. Li H, Boimel P, Janopaul-Naylor J, Zhong H, Xiao Y, BenJosef E, et al. Deep convolutional neural networks for imaging data based survival analysis of rectal cancer. Proc IEEE Int Symp Biomed Imaging. 2019;2019:846-849.
  33. Tait LM, Hoffman D, Benedict S, Valicenti R, Mayadev JS. The use of MRI deformable image registration for CT-based brachytherapy in locally advanced cervical cancer. Brachytherapy. 2016;15:333-340. https://doi.org/10.1016/j.brachy.2016.01.002
  34. Qi M, Li Y, Wu A, Jia Q, Li B, Sun W, et al. Multi-sequence MR image-based synthetic CT generation using a generative adversarial network for head and neck MRI-only radiotherapy. Med Phys. 2020;47:1880-1894. https://doi.org/10.1002/mp.14075
  35. Yang C, Liu F, Ahunbay E, Chang YW, Lawton C, Schultz C, et al. Combined online and offline adaptive radiation therapy: a dosimetric feasibility study. Pract Radiat Oncol. 2014;4:e75-e83. https://doi.org/10.1016/j.prro.2013.02.012
  36. Chen L, Liang X, Shen C, Jiang S, Wang J. Synthetic CT generation from CBCT images via deep learning. Med Phys. 2020;47:1115-1125. https://doi.org/10.1002/mp.13978
  37. Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation. Ithaca: arXiv.org, 2014 [cited 2020 Jul 23]. Available from: https://arxiv.org/abs/1411.4038.
  38. Zhou Z, He Z, Jia Y. AFPNet: A 3D fully convolutional neural network with atrous-convolution feature pyramid for brain tumor segmentation via MRI images. Neurocomputing. 2020;402:235-244. https://doi.org/10.1016/j.neucom.2020.03.097
  39. Ronneberger O, Fischer P, Brox T. U-net: convolutional networks for biomedical image segmentation. Ithaca: arXiv. org, 2015 [cited 2020 Jul 23]. Available from: https://arxiv.org/abs/1505.04597.
  40. Rachmadi MF, Valdes-Hernandez MDC, Agan MLF, Di Perri C, Komura T; Alzheimer's Disease Neuroimaging Initiative. Segmentation of white matter hyperintensities using convolutional neural networks with global spatial information in routine clinical brain MRI with none or mild vascular pathology. Comput Med Imaging Graph. 2018;66: 28-43. https://doi.org/10.1016/j.compmedimag.2018.02.002
  41. Zhu W, Huang Y, Zeng L, Chen X, Liu Y, Qian Z, et al. AnatomyNet: deep learning for fast and fully automated wholevolume segmentation of head and neck anatomy. Med Phys. 2019;46:576-589. https://doi.org/10.1002/mp.13300
  42. Ahn SH, Yeo AU, Kim KH, Kim C, Goh Y, Cho S, et al. Comparative clinical evaluation of atlas and deep-learningbased auto-segmentation of organ structures in liver cancer. Radiat Oncol. 2019;14:213. https://doi.org/10.1186/s13014-019-1392-z
  43. Quan TM, Hildebrand DGC, Jeong WK. FusionNet: a deep fully residual convolutional neural network for image segmentation in connectomics. Ithaca: arXiv.org, 2016 [cited 2020 Jul 23]. Available from: https://arxiv.org/abs/1612.05360.
  44. Vinod SK, Min M, Jameson MG, Holloway LC. A review of interventions to reduce inter-observer variability in volume delineation in radiation oncology. J Med Imaging Radiat Oncol. 2016;60:393-406. https://doi.org/10.1111/1754-9485.12462
  45. Rios Piedra EA, Taira RK, El-Saden S, Ellingson BM, Bui AAT, Hsu W. Assessing variability in brain tumor segmentation to improve volumetric accuracy and characterization of change. IEEE EMBS Int Conf Biomed Health Inform. 2016;2016:380-383.
  46. Apolle R, Appold S, Bijl HP, Blanchard P, Bussink J, Faivre-Finn C, et al. Inter-observer variability in target delineation increases during adaptive treatment of head-and-neck and lung cancer. Acta Oncol. 2019;58:1378-1385. https://doi.org/10.1080/0284186X.2019.1629017
  47. Chabane KTW. Interobserver variation of prostate delineation on CT and MR by radiation oncologists, radiologists and urologists at the Universitas annex oncology department [dissertation]. Bloemfontein: University of the Free State; 2018.
  48. Franco P, Arcadipane F, Trino E, Gallio E, Martini S, Iorio GC, et al. Variability of clinical target volume delineation for rectal cancer patients planned for neoadjuvant radio-therapy with the aid of the platform Anatom-e. Clin Transl Radiat Oncol. 2018;11:33-39. https://doi.org/10.1016/j.ctro.2018.06.002
  49. Cabrera GG, Ehrgott M, Mason AJ, Raith A. A matheuristic approach to solve the multiobjective beam angle optimization problem in intensity-modulated radiation therapy. 2018;25:243-268. https://doi.org/10.1111/itor.12241
  50. Breedveld S, Storchi PR, Voet PW, Heijmen BJ. iCycle: integrated, multicriterial beam angle, and profile optimization for generation of coplanar and noncoplanar IMRT plans. Med Phys. 2012;39:951-963. https://doi.org/10.1118/1.3676689
  51. Taasti VT, Hong L, Shim JSA, Deasy JO, Zarepisheh M. Automating proton treatment planning with beam angle selection using Bayesian optimization. Med Phys. 2020;47: 3286-3296. https://doi.org/10.1002/mp.14215
  52. Sadeghnejad Barkousaraie A, Ogunmolu O, Jiang S, Nguyen D. A fast deep learning approach for beam orientation optimization for prostate cancer treated with intensity-modulated radiation therapy. Med Phys. 2020;47:880-897. https://doi.org/10.1002/mp.13986
  53. Quan EM, Li X, Li Y, Wang X, Kudchadker RJ, Johnson JL, et al. A comprehensive comparison of IMRT and VMAT plan quality for prostate cancer treatment. Int J Radiat Oncol Biol Phys. 2012;83:1169-1178. https://doi.org/10.1016/j.ijrobp.2011.09.015
  54. Hoffmann M, Pacey J, Goodworth J, Laszcyzk A, Ford R, Chick B, et al. Analysis of a volumetric-modulated arc therapy (VMAT) single phase prostate template as a class solution. Rep Pract Oncol Radiother. 2019;24:92-96. https://doi.org/10.1016/j.rpor.2018.10.009
  55. Chen X, Men K, Li Y, Yi J, Dai J. A feasibility study on an automated method to generate patient-specific dose distributions for radiotherapy using deep learning. Med Phys. 2019; 46:56-64. https://doi.org/10.1002/mp.13262
  56. Barragan-Montero AM, Nguyen D, Lu W, Lin MH, NorouziKandalan R, Geets X, et al. Three-dimensional dose prediction for lung IMRT patients with deep neural networks: robust learning from heterogeneous beam configurations. Med Phys. 2019;46:3679-3691. https://doi.org/10.1002/mp.13597
  57. Galib SM, Lee HK, Guy CL, Riblett MJ, Hugo GD. A fast and scalable method for quality assurance of deformable image registration on lung CT scans using convolutional neural networks. Med Phys. 2020;47:99-109. https://doi.org/10.1002/mp.13890
  58. Nyf lot MJ, Thammasorn P, Wootton LS, Ford EC, Chaovalitwongse WA. Deep learning for patient-specific quality assurance: identifying errors in radiotherapy delivery by radiomic analysis of gamma images with convolutional neural networks. Med Phys. 2019;46:456-464. https://doi.org/10.1002/mp.13338
  59. Interian Y, Rideout V, Kearney VP, Gennatas E, Morin O, Cheung J, et al. Deep nets vs expert designed features in medical physics: an IMRT QA case study. Med Phys. 2018; 45:2672-2680. https://doi.org/10.1002/mp.12890
  60. Cheon W, Kim SJ, Hwang UJ, Min BJ, Han Y. Feasibility study of the fluence-to-dose network (FDNet) for patient-specific IMRT quality assurance. J Korean Phys Soc. 2019; 75:724-734. https://doi.org/10.3938/jkps.75.724
  61. Kim KH, Do WJ, Park SH. Improving resolution of MR images with an adversarial network incorporating images with different contrast. Med Phys. 2018;45:3120-3131. https://doi.org/10.1002/mp.12945
  62. Chun J, Zhang H, Gach HM, Olberg S, Mazur T, Green O, et al. MRI super-resolution reconstruction for MRI-guided adaptive radiotherapy using cascaded deep learning: in the presence of limited training data and unknown translation model. Med Phys. 2019;46:4148-4164. https://doi.org/10.1002/mp.13717
  63. Cheon W, Lee J, Min BJ, Han Y. Super-resolution model for high-precision in vivo proton range verification using a stereo gamma camera: a feasibility study. J Korean Phys Soc. 2019;75:617-627. https://doi.org/10.3938/jkps.75.617
  64. Lu Y, Kowarschik M, Huang X, Xia Y, Choi JH, Chen S, et al. A learning-based material decomposition pipeline for multi-energy x-ray imaging. Med Phys. 2019;46:689-703. https://doi.org/10.1002/mp.13317
  65. Cheon W, Kim SJ, Kim K, Lee M, Lee J, Jo K, et al. Feasibility of two-dimensional dose distribution deconvolution using convolution neural networks. Med Phys. 2019;46:5833-5847. https://doi.org/10.1002/mp.13869
  66. Grau C, Hoyer M. High-precision radiotherapy. Eur Oncol Rev. 2005:40-44.