The Transactions of the Korea Information Processing Society (정보처리학회 논문지)

Korea Information Processing Society (한국정보처리학회)
권호 표지
DOI QR코드

DOI QR Code

3DentAI: U-Nets for 3D Oral Structure Reconstruction from Panoramic X-rays

3DentAI: 파노라마 X-ray로부터 3차원 구강구조 복원을 위한 U-Nets

  • ;
  • 문성용 (조선대학교 치과대학 구강악안면외과 ) ;
  • 유원상 (선문대학교 정보통신공학과)
  • Received : 2024.06.13
  • Accepted : 2024.06.19
  • Published : 2024.07.31
Download PDF
⟨ Previous Next ⟩

Abstract

Extra-oral imaging techniques such as Panoramic X-rays (PXs) and Cone Beam Computed Tomography (CBCT) are the most preferred imaging modalities in dental clinics owing to its patient convenience during imaging as well as their ability to visualize entire teeth information. PXs are preferred for routine clinical treatments and CBCTs for complex surgeries and implant treatments. However, PXs are limited by the lack of third dimensional spatial information whereas CBCTs inflict high radiation exposure to patient. When a PX is already available, it is beneficial to reconstruct the 3D oral structure from the PX to avoid further expenses and radiation dose. In this paper, we propose 3DentAI - an U-Net based deep learning framework for 3D reconstruction of oral structure from a PX image. Our framework consists of three module - a reconstruction module based on attention U-Net for estimating depth from a PX image, a realignment module for aligning the predicted flattened volume to the shape of jaw using a predefined focal trough and ray data, and lastly a refinement module based on 3D U-Net for interpolating the missing information to obtain a smooth representation of oral cavity. Synthetic PXs obtained from CBCT by ray tracing and rendering were used to train the networks without the need of paired PX and CBCT datasets. Our method, trained and tested on a diverse datasets of 600 patients, achieved superior performance to GAN-based models even with low computational complexity.

파노라마 X-ray (PX) 및 Cone Beam Computed Tomography (CBCT)와 같은 구강 영상 기술은 영상 촬영 시 환자의 편의성과 전체 치아 정보를 시각화할 수 있는 능력으로 인해 치과 진료소에서 가장 선호되는 영상 기법이다. PX는 일상적인 임상 치료에 선호되고, CBCT는 복잡한 수술 및 임플란트 치료에 선호된다. 그러나 PX는 3차원 공간정보가 부족하다는 한계가 있는 반면 CBCT는 환자에게 높은 방사선 노출을 초래한다. PX가 이미 사용 가능한 경우 PX로부터 3D강 구강구조를 복원함으로써 추가 비용을 줄이고 방사선량을 피할 수 있다. 본 논문에서는 PX 이미지로부터 구강구조의 3차원 복원을 위한 U-Net 기반 딥러닝 프레임워크인 3DentAI를 제안한다. 제안된 프레임워크는 PX 이미지에서 깊이를 추정하기 위한 Attention U-Net 기반 재구성 모듈, 사전 정의된 초점 골 및 광선 데이터를 사용하여 예측된 편평 볼륨을 턱 모양에 정렬하기 위한 재정렬 모듈과, 구강의 원활한 표현을 얻기 위해 누락된 정보를 보간하는 3D U-Net 기반 개선 모듈의 세 가지 모듈로 구성된다. 네트워크를 훈련하기 위해, 쌍을 이루는 PX 및 CBCT 데이터셋 대신에 광선 추적 및 렌더링을 통해 CBCT로 부터 합성한 PX 데이터를 사용하였다. 600명의 환자로 구성된 다양한 데이터셋으로 모델을 훈련한 결과, 낮은 계산 복잡도에도 GAN 기반 모델에 비해 우수한 성능을 보였다.

Keywords

Acknowledgement

본 연구는 2022년도 교육부의 재원으로 한국연구재단의 지원을 받아 수행된 기본연구(2022R1 F1A1075204), 4단계 두뇌한국21 사업(4단계 BK21사업) 및 지자체-대학 협력기반 지역혁신사업(2022RIS-004), 중소기업벤처부의 재원으로 수행된 2021년도 창업성장기술개발사업(S3228660)의 연구결과로 수행되었음.

References

  1. A. G. Farman, Panoramic Corporation (Eds.), "Panoramic radiology: Seminars on maxillofacial imaging and interpretation," Springer, Berlin; New York, 2007. OCLC: ocn123114795. 
  2. R. Izzetti, M. Nisi, G. Aringhieri, L. Crocetti, F. Graziani, and C. Nardi, "Basic knowledge and new advances in panoramic radiography imaging techniques: A narrative review on what dentists and radiologists should know," Applied Sciences, Vol.11, No.17, pp.7858, 2021. 
  3. S. C. White and M. J. Pharoah (Eds.), "Oral radiology: Principles and interpretation," edition 7. ed., Elsevier, St. Louis, Missouri, 2014. OCLC: 862758150. 
  4. R. Jacobs, B. Salmon, M. Codari, B. Hassan, and M. M. Bornstein, "Cone beam computed tomography in implant dentistry: Recommendations for clinical use," BMC Oral Health, Vol.18, pp.88, 2018. 
  5. A. Al-Okshi, M. Nilsson, A. Petersson, M. Wiese, and C. Lindh, "Using GafChromic film to estimate the effective dose from dental cone beam CT and panoramic radiography," Dentomaxillofacial Radiology, Vol.42, No.추가, pp.20120343, 2013. 
  6. Y. Liang et al., "OralViewer: 3D Demonstration of Dental Surgeries for Patient Education with Oral Cavity Reconstruction from a 2D Panoramic X-ray," 26th International Conference on Intelligent User Interfaces, Association for Computing Machinery, New York, NY, USA, pp.553-563, 2021. 
  7. I. Goodfellow et al., "Generative adversarial nets, in Advances in Neural Information Processing Systems (NIPS)," Montreal, Canada, pp.2672-2680, 2014. 
  8. B. Mildenhall, P. P. Srinivasan, M. Tancik, J. T. Barron, R. Ramamoorthi, and R. Ng, "NeRF: Representing scenes as neural radiance fields for view synthesis," in: A. Vedaldi, H. Bischof, T. Brox, J.- M. Frahm (Eds.), Computer Vision - ECCV 2020, Springer International Publishing, Cham, pp.405-421, 2020.
  9. Y. Liang, W. Song, J. Yang, L. Qiu, K. Wang, and L. He, "X2Teeth: 3D Teeth Reconstruction from a Single Panoramic Radiograph," in: A.L. Martel, P. Abolmaesumi, D. Stoyanov, D. Mateus, M.A. Zuluaga, S.K. Zhou, D. Racoceanu, L. Joskowicz (Eds.), Medical Image Computing and Computer Assisted Intervention - MICCAI 2020, Springer International Publishing, Cham, pp.400-409, 2020. 
  10. W. Song, Y. Liang, J. Yang, K. Wang, and L. He, "Oral-3D: Reconstructing the 3D structure of oral cavity from panoramic X-ray," Proceedings of the AAAI Conference on Artificial Intelligence, Vol.35, No.1, pp.566-573, 2021. 
  11. W. Song, H. Zheng, D. Tu, C. Liang, and L. He, "Oral-3Dv2: 3D oral reconstruction from panoramic x-ray imaging with implicit neural representation," arXiv preprint arXiv: 2303.12123, 2023. 
  12. S. Park, S. Kim, D. Kwon, Y. Jang, I.-S. Song, and S. Baek, "NeBLa: Neural beer-lambert for 3D reconstruction of oral structures from panoramic radiographs," arXiv e-prints, arXiv-2304, 2023. 
  13. N. Burgos et al., "Iterative framework for the joint segmentation and CT synthesis of MR images: Application to MRI-only radiotherapy treatment planning," Physics in Medicine & Biology, Vol.62, No.11, pp.4237-4253, 2017.  https://doi.org/10.1088/1361-6560/aa66bf
  14. Y. Huang, L. Shao, and A. F. Frangi, "Simultaneous superresolution and cross-modality synthesis of 3d medical images using weakly-supervised joint convolutional sparse coding," In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, HI: IEEE, pp.5787-5796, 2017. 
  15. P. Henzler, V. Rasche, T. Ropinski, and T. Ritschel, "Single-image tomography: 3d volumes from 2d cranial x-rays," Computer Graphics Forum, Vol.37, No.2, pp.377-388, 2018.  https://doi.org/10.1111/cgf.13369
  16. X. Ying, H. Guo, K. Ma, J. Wu, Z. Weng, and Y. Zheng, "X2CT-GAN: reconstructing CT from biplanar X-rays with generative adversarial networks," In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.10619-10628, 2019.