IEMEK Journal of Embedded Systems and Applications (대한임베디드공학회논문지)

Institute of Embedded Engineering of Korea (대한임베디드공학회)
권호 표지
DOI QR코드

DOI QR Code

Efficient Semi-automatic Annotation System based on Deep Learning

  • Hyunseok Lee (Daegu Gyeongbuk Medical Innovation Foundation) ;
  • Hwa Hui Shin (Daegu Gyeongbuk Medical Innovation Foundation ) ;
  • Soohoon Maeng (Daegu Gyeongbuk Medical Innovation Foundation) ;
  • Dae Gwan Kim (Daegu Gyeongbuk Medical Innovation Foundation ) ;
  • Hyojeong Moon (Daegu Gyeongbuk Medical Innovation Foundation)
  • Received : 2023.09.18
  • Accepted : 2023.11.23
  • Published : 2023.12.31
Download PDF
⟨ Previous Next ⟩

Abstract

This paper presents the development of specialized software for annotating volume-of-interest on 18F-FDG PET/CT images with the goal of facilitating the studies and diagnosis of head and neck cancer (HNC). To achieve an efficient annotation process, we employed the SE-Norm-Residual Layer-based U-Net model. This model exhibited outstanding proficiency to segment cancerous regions within 18F-FDG PET/CT scans of HNC cases. Manual annotation function was also integrated, allowing researchers and clinicians to validate and refine annotations based on dataset characteristics. Workspace has a display with fusion of both PET and CT images, providing enhance user convenience through simultaneous visualization. The performance of deeplearning model was validated using a Hecktor 2021 dataset, and subsequently developed semi-automatic annotation functionalities. We began by performing image preprocessing including resampling, normalization, and co-registration, followed by an evaluation of the deep learning model performance. This model was integrated into the software, serving as an initial automatic segmentation step. Users can manually refine pre-segmented regions to correct false positives and false negatives. Annotation images are subsequently saved along with their corresponding 18F-FDG PET/CT fusion images, enabling their application across various domains. In this study, we developed a semi-automatic annotation software designed for efficiently generating annotated lesion images, with applications in HNC research and diagnosis. The findings indicated that this software surpasses conventional tools, particularly in the context of HNC-specific annotation with 18F-FDG PET/CT data. Consequently, developed software offers a robust solution for producing annotated datasets, driving advances in the studies and diagnosis of HNC.

Keywords

Acknowledgement

This research was supported by core technology development project for reorganization of new industries through the Korea Institute for Advancement of Technology (KIAT) funded by the Ministry of Trade, Industry and Energy (MOTIE) (grant number: P0018663).

References

  1. S. N. Histed, M. L. Lindenberg, E. Mena, B. Turkbey, P. L. Choyke, K. A. Kurdziel, "Review of Functional/anatomic Imaging in Oncology," Nucl. Med. Commun., Vol. 33, No. 4, pp. 349-361, 2012.  https://doi.org/10.1097/MNM.0b013e32834ec8a5
  2. K. Wechalekar, B. Sharma, G. Cook, "PET/CT in Oncology-a Major Advance," Clin. Radiol., Vol. 60, No. 11, pp. 1143-1155, 2005.  https://doi.org/10.1016/j.crad.2005.05.018
  3. B. J. Krause, M. Souvatzoglou, U. Treiber, "Imaging of Prostate Cancer with PET/CT and Radioactively Labeled Choline Derivates," Urol. Oncol-Semin. Ori., Vol. 31, No. 4, pp. 427-435, 2013.  https://doi.org/10.1016/j.urolonc.2010.08.008
  4. N. Avril, M. Menzel, J. Dose, M. Schelling, W. Weber, F. Janicke, W. Nathrath, M. Schwaiger, "Glucose Metabolism of Breast Cancer Assessed by 18F-FDG PET: Histologic and Immunohistochemical Tissue Analysis," J. Nucl. Med., Vol. 42, No. 1, pp. 9-16, 2001. 
  5. S. Rege, A. Maass, L. Chaiken, C. K. Hoh, Y. Choi, R. Lufkin, Y. Anzai, G. Juillard, J. Maddahi, M. E. Phelps, "Use of Positron Emission Tomography with Fluorodeoxyglucose in Patients with Extracranial Head and Neck Cancers," Cancer, Vol. 73, No. 12, pp. 3047-3058, 1994.  https://doi.org/10.1002/1097-0142(19940615)73:12<3047::AID-CNCR2820731225>3.0.CO;2-#
  6. J. W. Braams, J. Pruim, N. J. Freling, P. G. Nikkels, J. L. Roodenburg, G. Boering, W. Vaalburg, A. Vermey, "Detection of Lymph Node Metastases of Squamous-cell Cancer of the Head and Neck with FDG-PET and MRI," J. Nucl. Med., Vol. 36, No. 2, pp. 211-216, 1995. 
  7. W. Halfpenny, S. F. Hain, L. Biassoni, M. N. Maisey, J. A. Sherman, M. McGurk, "FDG-PET. A Possible Prognostic Factor in Head and Neck Cancer," Brit. J. Cancer., Vol. 86, No. 4, pp. 512-516, 2002.  https://doi.org/10.1038/sj.bjc.6600114
  8. J. A. Bonner, P. M. Harari, J. Giralt, R. B. Cohen, C. U. Jones, R. K. Sur, D. Raben, J. Baselga, S. A. Spencer, J. Zhu, H. Youssoufian, E. K. Rowinsky, K. K. Ang, "Radiotherapy Plus Cetuximab for Locoregionally Advanced Head and Neck Cancer: 5-year Survival Data from a Phase 3 Randomised Trial, and Relation Between Cetuximab-induced Rash and Survival," Lancet. Oncol., Vol. 11, No. 1, pp. 21-28, 2010.  https://doi.org/10.1016/S1470-2045(09)70311-0
  9. V. Kumar, Y. Gu, S. Basu, A. Berglund, S. A. Eschrich, M. B. Schabath, K. Forster, H. J. Aerts, A. Dekker, D. Fenstermacher, D. G. Goldgof, L. O. Hall, P. Lambin, Y. Balagurunathan, R. A. Gatenby, R. J. Gillies, "Radiomics: the Process and the Challenges," Magn. Reson. Imaging., Vol. 30, No. 9, pp. 1234-1248, 2012.  https://doi.org/10.1016/j.mri.2012.06.010
  10. S. Gatidis, T. Hepp, M. Fruh, C. L. Fougere, K. Nikolaou, C. Pfannenberg, B. Scholkopf, T. Kustner, C. Cyran, D. Rubin, "A Whole-body FDG-PET/CT Dataset with Manually Annotated Tumor Lesions," Sci. Data., Vol. 9, No. 1, pp. 601, 2022. 
  11. S. Yu, M. Chen, E. Zhang, J. Wu, H. Yu, Z. Yang, L. Ma, X. Gu, W. Lu, "Robustness Study of Noisy Annotation in Deep Learning Based Medical Image Segmentation," Phys. Med. Biol., Vol. 65, No. 17, pp. 175007, 2020. 
  12. K. A. Philbrick, A. D. Weston, Z. Akkus, T. L. Kline, P. Korfiatis, T. Sakinis, P. Kostandy, A. Boonrod, A. Zeinoddini, N. Takahashi, B. J. Erickson, "RIL-contour: a Medical Imaging Dataset Annotation Tool for and with Deep Learning." J. Digit. Imaging, Vol. 32 pp. 571-581, 2019.  https://doi.org/10.1007/s10278-019-00232-0
  13. V. Oreiller, V. Andrearczyk, M. Jreige, S. Boughdad, H. Elhalawani, J. Castelli, M. Vallieres, S. Zhuf, J. Xie, Y. Peng, A. Iantsenh, M. Hatt, Y. Yuan, J. Ma, X. Yang, C. Rao, S. Pai, K. Ghimire, X. Feng, M. A. Naser, C. D. Fuller, F. Yousefirizi, A. Rahmim, H. Chen, L. Wang, J. O. Prior, A. Depeursinge, "Head and Neck Tumor Segmentation in PET/CT: the HECKTOR Challenge," Med. image. anal., Vol. 77, pp. 102336, 2022. 
  14. V. Andrearczyk, V. Oreiller, S. Boughdad, C. C. L. Rest, H. Elhalawani, M. Jreige, J. O. Prior, M. Valli'eres, D. Visvikis, M. Hatt, A. Depeursinge, "Overview of the HECKTOR Challenge at MICCAI 2021: Automatic Head and Neck Tumor Segmentation and Outcome Prediction in PET/CT Images," 3D Head and Neck Tumor Segmentation in PET/CT Challenge, pp. 1-37, 2021. 
  15. J. Xie, Y. Peng, M. Wang, "The Squeeze & Excitation Normalization based nnU-Net for Segmenting Head & Neck Tumors," Chinese. J. Electron., Vol. 33, pp. 1-11, 2022. 
  16. Y. Yuan, "Automatic Head and Neck Tumor Segmentation in PET/CT with Scale Attention Network," Head and Neck Tumor Segmentation: First Challenge, HECKTOR 2020, Held in Conjunction with MICCAI 2020, Lima, Peru, October 4, 2020, Proceedings, pp.44-52, 2021. 
  17. A. Iantsen, D. Visvikis, M. Hatt, "Squeeze-and-excitation Normalization for Automated Delineation of Head and Neck Primary Tumors in Combined PET and CT Images," Head and Neck Tumor Segmentation: First Challenge, HECKTOR 2020, Held in Conjunction with MICCAI 2020, Lima, Peru, October 4, 2020, Proceedings, pp.37-43, 2021. 
  18. B. Hariharan, P. Arbelaez, R. Girshick, J. Malik, "Hypercolumns for Object Segmentation and Fine-grained Localization," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 447-456, 2015. 
  19. P. A. Yushkevich, Y. Gao, G. Gerig, "ITK-SNAP: An Interactive Tool for Semi-automatic Segmentation of Multi-modality Biomedical Images," 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 3342-3345, 2016. 
  20. H. Zhao, J. Shi, X. Qi, X. Wang, J. Jia, "Pyramid Scene Parsing Network," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2881-2890, 2017. 
  21. T. Lin, P. Dollar, R. Girshick, K. He, B. Hariharan, S. Belongie, "Feature Pyramid Networks for Object Detection," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2117-2125, 2017. 
  22. J. Chen, Y. Lu, Q. Yu, X. Luo, E. Adeli, Y. Wang, L. Lu, A. L. Yuille, Y. Zhou, "Transunet: Transformers Make Strong Encoders for Medical Image Segmentation," arXiv preprint arXiv:2102.04306, 2021. 
  23. H. Lee, H. Shin, G. S. Choi, S. Jin, "Performance Analysis of Deep Learning-based Image Super Resolution Methods," IEMEK J. Embed. Sys. Appl., Vol. 15, No. 2, pp. 61-70, 2020.