Journal of Biomedical Engineering Research (대한의용생체공학회:의공학회지)

The Korean Society of Medical and Biological Engineering (대한의용생체공학회)
권호 표지
DOI QR코드

DOI QR Code

3D Ultrasound Panoramic Image Reconstruction using Deep Learning

딥러닝을 활용한 3차원 초음파 파노라마 영상 복원

  • SiYeoul Lee (Department of Information Convergence Engineering, College of Information and BioMedical Engineering, Pusan National University) ;
  • Seonho Kim (Department of Information Convergence Engineering, College of Information and BioMedical Engineering, Pusan National University) ;
  • Dongeon Lee (Department of Information Convergence Engineering, College of Information and BioMedical Engineering, Pusan National University) ;
  • ChunSu Park (School of Biomedical Convergence Engineering, College of Information and BioMedical Engineering, Pusan National University) ;
  • MinWoo Kim (School of Biomedical Convergence Engineering, College of Information and BioMedical Engineering, Pusan National University)
  • 이시열 (부산대학교 정보융합공학과) ;
  • 김선호 (부산대학교 정보융합공학과) ;
  • 이동언 (부산대학교 정보융합공학과) ;
  • 박춘수 (부산대학교 의생명융합공학부) ;
  • 김민우 (부산대학교 의생명융합공학부)
  • Received : 2023.07.16
  • Accepted : 2023.08.04
  • Published : 2023.08.31
Download PDF
⟨ Previous Next ⟩

Abstract

Clinical ultrasound (US) is a widely used imaging modality with various clinical applications. However, capturing a large field of view often requires specialized transducers which have limitations for specific clinical scenarios. Panoramic imaging offers an alternative approach by sequentially aligning image sections acquired from freehand sweeps using a standard transducer. To reconstruct a 3D volume from these 2D sections, an external device can be employed to track the transducer's motion accurately. However, the presence of optical or electrical interferences in a clinical setting often leads to incorrect measurements from such sensors. In this paper, we propose a deep learning (DL) framework that enables the prediction of scan trajectories using only US data, eliminating the need for an external tracking device. Our approach incorporates diverse data types, including correlation volume, optical flow, B-mode images, and rawer data (IQ data). We develop a DL network capable of effectively handling these data types and introduce an attention technique to emphasize crucial local areas for precise trajectory prediction. Through extensive experimentation, we demonstrate the superiority of our proposed method over other DL-based approaches in terms of long trajectory prediction performance. Our findings highlight the potential of employing DL techniques for trajectory estimation in clinical ultrasound, offering a promising alternative for panoramic imaging.

Keywords

Acknowledgement

본 연구는 2020학년도 부산대학교 교내학술연구비(신임교수연구정착금)에 의한 연구임. 본 연구는 과학기술정보통신부 및 정보통신기획평가원의 인공지능융합혁신인재양성사업 연구 결과로 수행되었음 (IITP-2023-RS-2023-00254177). 이 논문은 정부(과학기술정보통신부)의 재원으로 한국연구재단의 지원을 받아 수행된 연구임 (No. 2021R1A2C2094778)

References

  1. Mi J, Fang Y, Xian J, Wang G, Guo Y, Hong H, Chi M, Li YF, He P, Gao J, Liao W. "Comparative effectiveness of MRI, 4D-CT and ultrasonography in patients with secondary hyperparathyroidism". Therapeutics and Clinical Risk Management, 2023;19:369-81. https://doi.org/10.2147/TCRM.S379814
  2. Chen JF, Fowlkes JB, Carson PL, Rubin JM. "Determination of scan-plane motion using speckle decorrelation: Theoretical considerations and initial test". International Journal of Imaging Systems and Technology. 1997;8(1):38-44. https://doi.org/10.1002/(SICI)1098-1098(1997)8:1<38::AID-IMA5>3.0.CO;2-U
  3. Tuthill TA, Krucker JF, Fowlkes JB, Carson PL. "Automated three-dimensional US frame positioning computed from elevational speckle decorrelation". Radiology. 1998;209(2):575-82. https://doi.org/10.1148/radiology.209.2.9807593
  4. Mondillo S, Galderisi M, Mele D, Cameli M, Lomoriello VS, Zaca V, Ballo P, Andrea A, Muraru D, Losi M, Agricola E, Errico A, Buralli S, Sciomer S, Nistri S, Badano L. "Speckle-tracking echocardiography: a new technique for assessing myocardial function". Journal of Ultrasound in Medicine. 2011;30(1):71-83. https://doi.org/10.7863/jum.2011.30.1.71
  5. Ronneberger O, Philipp F, Thomas B. "U-net: Convolutional networks for biomedical image segmentation". Medical Image Computing and Computer-Assisted Intervention-MICCAI 2015. 2015;234-41.
  6. Esteva A, Kuprel B, Novoa RA, KO J, Swetter SM, Blau HM, Thrun S. "Dermatologist-level classification of skin cancer with deep neural networks". Nature. 2017;542:115-18. https://doi.org/10.1038/nature21056
  7. Gulshan V, Peng L, Coram M, Stumpe MC, Wu D, Narayanaswamy A, Venugopalan S, Widner K, Madams T, Cuadros J, Kim R, Raman R, Nelson PC, Mega JL, Webster DR. "Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs". JAMA. 2016;316(22):2402-10. https://doi.org/10.1001/jama.2016.17216
  8. Prevost R, Salehi M, Jagoda S, Kumer N, Sprung J, Ladikos A, Bauer R, Zettinig O, Wein W. "3D freehand ultrasound without external tracking using deep learning". Medical image analysis. 2018;48:187-202. https://doi.org/10.1016/j.media.2018.06.003
  9. Guo H, Xu S, Wood B, Yan P. "Sensorless freehand 3D ultrasound reconstruction via deep contextual learning". Medical Image Computing and Computer Assisted Intervention-MICCAI 2020. 2020;463-72.
  10. Teed Z, Deng J. "Raft: Recurrent all-pairs field transforms for optical flow". Computer Vision-ECCV 2020. 2020; 402-19.
  11. He K, Zhang X, Ren S, Sun J. "Deep residual learning for image recognition". IEEE conference on computer vision and pattern recognition (CVPR). 2016;770-78.
  12. Woo SH, Park J, Lee JY, Kweon IS. "Cbam: Convolutional block attention module". European conference on computer vision (ECCV). 2018;3-19.