Journal of the Institute of Convergence Signal Processing (융합신호처리학회논문지)

The Korea Institute of Convergence Signal Processing (한국융합신호처리학회)
권호 표지
DOI QR코드

DOI QR Code

Deep Learning Based Digital Staining Method in Fourier Ptychographic Microscopy Image

Fourier Ptychographic Microscopy 영상에서의 딥러닝 기반 디지털 염색 방법 연구

  • 황석민 (계명대학교 의과대학 의학과 의용공학전공) ;
  • 김동범 (계명대학교 의과대학 의용공학과) ;
  • 김유정 (계명대학교 의과대학 의용공학과) ;
  • 김여린 (계명대학교 의과대학 의용공학과) ;
  • 이종하 (계명대학교 의과대학 의용공학과)
  • Received : 2022.06.07
  • Accepted : 2022.06.23
  • Published : 2022.06.30
Download PDF
⟨ Previous Next ⟩

Abstract

In this study, H&E staining is necessary to distinguish cells. However, dyeing directly requires a lot of money and time. The purpose is to convert the phase image of unstained cells to the amplitude image of stained cells. Image data taken with FPM was created with Phase image and Amplitude image using Matlab's parameters. Through normalization, a visually identifiable image was obtained. Through normalization, a visually distinguishable image was obtained. Using the GAN algorithm, a Fake Amplitude image similar to the Real Amplitude image was created based on the Phase image, and cells were distinguished by objectification using MASK R-CNN with the Fake Amplitude image As a result of the study, D loss max is 3.3e-1, min is 6.8e-2, G loss max is 6.9e-2, min is 2.9e-2, A loss max is 5.8e-1, min is 1.2e-1, Mask R-CNN max is 1.9e0, and min is 3.2e-1.

본 연구에서 세포를 분별하기 위해 H&E 염색이 필요하다. 그러나 직접 염색하면 많은 비용과 시간이 필요하다. H&E 염색되지 않은 세포의 Phase image에서 H&E 염색이 된 세포의 Amplitude image로 변환 하는 것이 목적이다. FPM으로 촬영한 Image data를 가지고 Matlab을 이용해 매개변수를 변경해 Phase image와 Amplitude image를 만들었다. 정규화를 통해 육안으로 식별이 가능한 이미지를 얻었다. GAN 알고리즘을 이용해 Phase image를 기반으로 Real Amplitude image와 비슷한 Fake Amplitude image를 만들고 Fake Amplitude image를 가지고 MASK R-CNN을 이용하여 세포를 분별하여 객체화를 통해 구분했다. 연구 결과 D loss의 max는 3.3e-1, min은 6.8e-2, G loss max는 6.9e-2, min은 2.9e-2, A loss는 max 5.8e-1, min은 1.2e-1, Mask R-CNN max는 1.9e0, min은 3.2e-1이다.

Keywords

Acknowledgement

본 연구는 한국연구재단 지역대학우수과학자과제의 지원으로 수행되었음(NRF-2022R1I1A3072785).

References

  1. H Hwang. "Future risk factors of artificial intelligence technology and the direction of our response." Collection of materials for academic conferences of the Korean Local Government Society, 2021, pp.49-81.
  2. SJ YU "4th Industrial Revolution and Artificial Intelligence.", Korean Multimedia Society, Volume 21, no. 4, 2017. pp.1-8.
  3. Mayerich. Author et al,"Bhargava, Rohit Stain-less staining for computed histopathology", Technology, vol.3, no.1, 2015, pp.27-31. https://doi.org/10.1142/S2339547815200010
  4. JT Gwak. "Digital Pathology Technology Status and Development Direction.", Electronic Engineering Society, Volume 44, no. 2, 2017, pp.36-41.
  5. JW Oh. Author et al. "Improvement activities of the examination process to improve the quality of general biopsy (H&E Stay).", Tissue Cell Examination Society abstract, Volume 2010, no. 2, 2010, pp.2-3.
  6. Faliu Yi. Author et al "Automatic extraction of cell nuclei from H&E-stained histopathological images", Journal of Medical Imaging, vol.4, no.2, 2017, pp.1-12.
  7. de Haan. Author et al. "Deep learning-based transformation of H&E stained tissues into special stains.", Nature Communications, vol.12, no.1, 2021(12), pp.1-13. https://doi.org/10.1038/s41467-020-20314-w
  8. Avwioro, G. "Histochemical Uses of Haematoxylin - A Review", JPCS, vol.1, 2011, pp.24-34.
  9. YH Oh. "The outlook for supply and demand and policy direction of clinical pathologists.", Korean Society of Clinical Laboratory, vol. 50, no. 4, 2018, pp.511-524. https://doi.org/10.15324/kjcls.2018.50.4.511
  10. JT Kwak. "Digital Pathology Technology Status and Development Direction.", Electronic Engineering Society, v 44, no. 2, 2017, pp.36-41.
  11. Marshall PN, Horobin RW. "The oxidation products of haematoxylin and their role in biological staining", The Histochemical Journal. vol.4, 1972, pp.493-503. https://doi.org/10.1007/BF01011129
  12. Clark George. "Comparison of various oxidants for alum hematoxylin", Stain Technology, vol.49, no.4, 1974, pp.225-227. https://doi.org/10.3109/10520297409116982
  13. Kulkarni. Author et al, (2020) "Deep Learning Based on Standard H&E Images of Primary Melanoma Tumors Identifies Patients at Risk for Visceral Recurrence and Death", Clinical cancer research : an official journal of the American Association for Cancer Research.
  14. SY Song, Introduction to Pathological Artificial Intelligence, Hanyang Medical Reviews, 2017, pp77-85.
  15. Kyoohyun Kim, Jochen Guck. "The Relative Densities of Cytoplasm and Nuclear Compartments Are Robust against Strong Perturbation", Biophysical Journal, vol.119, no.10, 2020, pp.1946-1957.
  16. IJ. Goodfellow Author et al, "Generative Adversarial Networks", NIPS, 2014.
  17. K. He Author et al, "Mask R-CNN", Proceedings of the IEEE International Conference on Computer Vision(ICCV), 2017, pp.2961-2969.
  18. J. Hosang, R. Benenson, B. Schiele. "What makes for effective detection proposals?", PAMI, 2015.
  19. JRR Uijlings, K. E. A. van de Sande, T. Gevers, A. W. M, "Selective search for object recognition." IJCV, 2013.
  20. Alex Krizhevsky . Ilya Sutskever . Geoffrey E. Hinton, "ImageNet classification with deep convolutional neural networks", NIPS, 2012.
  21. Y. LeCun Author et al, "Backpropagation appFaked to handwritten zip code recognition", Neural computation, 1989.
  22. J. Huang Author et al. "Speed/accuracy trade-offs for modern convolutional object detectors", CVPR, 2017.
  23. R. Girshick. "Fast R-CNN", ICCV, 2015.
  24. S. M .Hwang, S. W. Lee, J. H. Lee. "A Study on a Mask R-CNN-Based Diagnostic System Measuring DDH Angles on Ultrasound Scans," JISPS, vol.21, no.4, pp.183-194, 2020.
  25. C Zheng, "Robust Fourier ptychographic microscopy via a physics-based defocusing strategy for calibrating angle-varied LED illumination", Biomedical Optics Express, vol. 13, Issue 3, 2022, pp. 1581-1594. https://doi.org/10.1364/BOE.452507
  26. Y Rivenson Author et al, "PhaseStain: the digital staining of label-free quantitative phase microscopy images using deep learning", Light: Science & Applications 8, Article number: 23, 2019(February), pp.2-11. https://doi.org/10.1038/s41377-018-0112-z
  27. F. Zernike."Phase contrast, a new method for the microscopic observation of transparent objects", Physica, vol.9, no.7, 1942, pp.686-698.  https://doi.org/10.1016/S0031-8914(42)80035-X