Journal of the Korea Computer Graphics Society (한국컴퓨터그래픽스학회논문지)

Korea Computer Graphics Society (한국컴퓨터그래픽스학회)
권호 표지
DOI QR코드

DOI QR Code

TF color mapping for direct volume rendering with CNN

직접 볼륨 렌더링을 위한 CNN 기반 TF 색상 매핑

  • Kim, Seokyeon (Department of Computer Engineering, Sejong University) ;
  • Jang, Yun (Department of Computer Engineering, Sejong University)
  • 김석연 (세종대학교 컴퓨터공학과) ;
  • 장윤 (세종대학교 컴퓨터공학과)
  • Received : 2021.08.17
  • Accepted : 2021.11.11
  • Published : 2021.12.01
Download PDF
⟨ Previous Next ⟩

Abstract

Direct Volume Rendering(DVR) renders by projecting data into a two-dimensional space without calculating the volume surfaces. In DVR, the transfer function(TF) assigns light properties such as color and transparency to the volume. However, it takes a long time for beginners to manipulate TF to understand volume data and assign colors. This paper proposes an approach to colorize the volume using sample images for intuitive volume rendering. We also discuss color extraction methods using K-means clustering.

직접 볼륨 렌더링은 볼륨 표면의 연산 없이 2차원 공간에 투영하여 렌더링 한다. 직접 볼륨 렌더링에서 전이함수(TF)는 볼륨에 색상과 투명도와 같은 광원 특성을 할당하는데 활용된다. 하지만 초보자가 TF를 조작하여 볼륨데이터를 파악하고 색상을 할당하기까지 오랜 시간이 필요합니다. 본 논문에서는 직관적인 볼륨 렌더링을 위해 인터넷에서 수집한 이미지를 사용하여 TF에 볼륨의 색상을 매핑하는 접근 방식을 제안한다. 또한 우리는 K-means 클러스터링을 활용한 색상 추출 방법을 토의한다.

Keywords

Acknowledgement

이 논문은 2021년도 정부(과학기술정보통신부)의 재원으로 정보통신기획평가원의 지원을 받아 수행된 연구임 (No. 2021-0-00469, 이종 융합 데이터 탐지 및 추적 기술 개발)(No. 2021-0-02076, 빅데이터 인과 분석을 위한 복잡계 기반 추론 인공지능 개발 및 실증)

References

  1. J. Kniss, G. Kindlmann, and C. Hansen. "Multidimensional transfer functions for interactive volume rendering." IEEE Transactions on Visualization and Computer Graphics, 8(3):270-285, 2002. https://doi.org/10.1109/TVCG.2002.1021579
  2. S. Arens and G. Domik, "A survey of transfer functions suitable for volume rendering," in Proceedings of the 8th IEEE/EG international conference on Volume Graphics, pp. 77-83, Eurographics Association, 2010.
  3. P. Ljung, J. Kruger, E. Groller, M. Hadwiger, C. D. Hansen, and A. Ynnerman, "State of the art in transfer functions for direct volume rendering," Computer Graphics Forum, vol. 35, no. 3, pp. 669-691, 2016. https://doi.org/10.1111/cgf.12934
  4. R. Maciejewski, Y. Jang, I.Woo, H. Janicke, K. P. Gaither, and D. S. Ebert, "Abstracting attribute space for transfer function exploration and design," IEEE Transactions on Visualization and Computer Graphics, vol. 19, no. 1, pp. 94-107, 2013. https://doi.org/10.1109/TVCG.2012.105
  5. S. Kim, Y. Jang, SE. Kim, "Image-Based TF Colorization With CNN for Direct Volume Rendering" IEEE Access, 9, 124281-124294, 2021. https://doi.org/10.1109/ACCESS.2021.3100429
  6. R. A. Drebin, L. Carpenter, and P. Hanrahan. "Volume rendering." In Proceedings of the 15th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH '88, pp. 65-74. ACM, New York, NY, USA, 1988.
  7. H. Childs, E. Brugger, B. Whitlock, J. Meredith, S. Ahern, D. Pugmire, K. Biagas, M. Miller, G. H. Weber, H. Krishnan, et al. "Visit: An enduser tool for visualizing and analyzing very large data." High performance visualization-enabling extreme-scale scientific insight, pp. 357-372, 2012.
  8. T. Fogal and J. Kruger. Tuvok, "an Architecture for Large Scale Volume Rendering." In Proceedings of the 15th International Workshop on Vision, Modeling, and Visualization, November 2010.
  9. T. He, L. Hong, A. Kaufman, and H. Pfister. "Generation of transfer functions with stochastic search techniques." In Proceedings of Seventh Annual IEEE Visualization'96, pp. 227-234. IEEE, 1996.
  10. J. Li, L. Zhou, H. Yu, H. Liang, and L. Wang. "Classification for volume rendering of industrial ct based on moment of histogram." In 2007 2nd IEEE Conference on Industrial Electronics and Applications, pp. 913-918, 2007
  11. C. Lundstrom, P. Ljung, and A. Ynnerman. "Local histograms for design of transfer functions in direct volume rendering." IEEE Transactions on Visualization and Computer Graphics, 12(6):1570-1579, 2006. https://doi.org/10.1109/TVCG.2006.100
  12. S. Martin, S. Bruton, D. Ganter, and M. Manzke. "Using a depth heuristic for light field volume rendering." In Proceedings of the 14th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 1: GRAPP,, pp. 134-144, 2019.
  13. J. Hladuvka, A. Konig, and E. Groller. "Curvature-based transfer functions for direct volume rendering." In Proceedings of Spring Conference on Computer Graphics and its Applications, pp. 58-65, 2000.
  14. G. Kindlmann and J. W. Durkin. "Semi-automatic generation of transfer functions for direct volume rendering." In Volume Visualization, 1998. IEEE Symposium on, pp. 79-86. IEEE, 1998.
  15. G. Kindlmann, R. Whitaker, T. Tasdizen, and T. Moller. "Curvature-based transfer functions for direct volume rendering: Methods and applications." In Visualization, 2003. VIS 2003. IEEE, pp. 513-520. IEEE, 2003.
  16. S. Roettger, M. Bauer, and M. Stamminger. "Spatialized transfer functions." In EuroVis, pp. 271-278, 2005.
  17. C. Correa and K.-L. Ma. "The occlusion spectrum for volume classification and visualization." IEEE Transactions on Visualization and Computer Graphics, 15(6):1465-1472, 2009. https://doi.org/10.1109/TVCG.2009.189
  18. C. D. Correa and K.-L. Ma. "Visibility histograms and visibility-driven transfer functions." IEEE Transactions on Visualization and Computer Graphics, 17(2):192-204, 2011. https://doi.org/10.1109/TVCG.2010.35
  19. J. M. Kniss, R. V. Uitert, A. Stephens, G. . Li, T. Tasdizen, and C. Hansen. "Statistically quantitative volume visualization." In VIS 05. IEEE Visualization, 2005., pp. 287-294, 2005.
  20. P. Sereda, A. V. Bartroli, I. W. Serlie, and F. A. Gerritsen. "Visualization of boundaries in volumetric data sets using lh histograms." IEEE Transactions on Visualization and Computer Graphics, 12(2):208-218, 2006. https://doi.org/10.1109/TVCG.2006.39
  21. M. A. Selver and C. Guzelis. "Semiautomatic transfer function initialization for abdominal visualization using self-generating hierarchical radial basis function networks." IEEE Transactions on Visualization and Computer Graphics, 15(3):395-409, 2009. https://doi.org/10.1109/TVCG.2008.198
  22. R. Maciejewski, I. Woo, W. Chen, and D. Ebert. "Structuring feature space: A non-parametric method for volumetric transfer function generation." IEEE Transactions on Visualization and Computer Graphics, 15(6):1473-1480, 2009. https://doi.org/10.1109/TVCG.2009.185
  23. M. Haidacher, D. Patel, S. Bruckner, A. Kanitsar, and M. E. Groller. "Volume visualization based on statistical transfer-function spaces." In IEEE Pacific Visualization Symposium (PacificVis), pp. 17-24, 2010.
  24. W. Serna-Serna, A. M. Alvarez-Meza, and A. A. Orozco-Gutierrez. "Volume rendering by stochastic neighbor embedding-based 2d transfer function building." In Iberoamerican Congress on Pattern Recognition, pp. 618-626. Springer, 2017.
  25. S. Castro, A. Konig, H. Loffelmann, and E. Groller. "Transfer function specification for the visualization of medical data." Vienne University of Technology, 1998.
  26. C. R. Salama, M. Keller, and P. Kohlmann. "High-level user interfaces for transfer function design with semantics." IEEE Transactions on Visualization and Computer Graphics, 12(5), 2006.
  27. C. Lundstrom, P. Ljung, and A. Ynnerman. "Multi-Dimensional Transfer Function Design Using Sorted Histograms." In R. Machiraju and T. Moeller, eds., Volume Graphics. The Eurographics Association, 2006.
  28. Y. Wu and H. Qu. "Interactive transfer function design based on editing direct volume rendered images." IEEE Transactions on Visualization and Computer Graphics, 13(5):1027-1040, 2007. https://doi.org/10.1109/TVCG.2007.1051
  29. J. J. Caban and P. Rheingans. "Texture-based transfer functions for direct volume rendering." IEEE Transactions on Visualization and Computer Graphics, 14(6):1364-1371, 2008. https://doi.org/10.1109/TVCG.2008.169
  30. S. Wesarg, M. Kirschner, and M. F. Khan. "2d histogram based volume visualization: combining intensity and size of anatomical structures." International journal of computer assisted radiology and surgery, 5(6):655-666, 2010. https://doi.org/10.1007/s11548-010-0480-1
  31. C. Correa and K.-L. Ma. "Size-based transfer functions: A new volume exploration technique." IEEE Transactions on Visualization and Computer Graphics, 14(6):1380-1387, 2008. https://doi.org/10.1109/TVCG.2008.162
  32. H. Akibay and K.-L. May. "A tri-space visualization interface for analyzing time-varying multivariate volume data." In Proceedings of the 9th Joint Eurographics/IEEE VGTC conference on Visualization, pp. 115-122. Eurographics Association, 2007.
  33. H. Guo, H. Xiao, and X. Yuan. "Multi-dimensional transfer function design based on flexible dimension projection embedded in parallel coordinates." In IEEE Pacific Visualization Symposium (PacificVis), pp. 19-26, 2011.
  34. H. Guo, H. Xiao, and X. Yuan. "Scalable multivariate volume visualization and analysis based on dimension projection and parallel coordinates." IEEE Transactions on Visualization and Computer Graphics, 18(9):1397-1410, 2012. https://doi.org/10.1109/TVCG.2012.80
  35. H. Guo, N. Mao, and X. Yuan. "Wysiwyg (what you see is what you get) volume visualization." IEEE Transactions on Visualization and Computer Graphics, 17(12):2106-2114, 2011. https://doi.org/10.1109/TVCG.2011.261
  36. L. Wang, X. Zhao, and A. E. Kaufman. "Modified dendrogram of attribute space for multidimensional transfer function design." IEEE Transactions on Visualization and Computer Graphics, 18(1):121-131, 2012. https://doi.org/10.1109/TVCG.2011.23
  37. S. Lan, L. Wang, Y. Song, Y.-p. Wang, L. Yao, K. Sun, B. Xia, and Z. Xu. "Improving separability of structures with similar attributes in 2d transfer function design." IEEE Transactions on Visualization and Computer Graphics, 23(5):1546-1560, 2017. https://doi.org/10.1109/TVCG.2016.2537341
  38. I. Fujishiro, T. Azuma, and Y. Takeshima. "Automating transfer function design for comprehensible volume rendering based on 3d field topology analysis." In Visualization'99. Proceedings, pp. 467-563. IEEE, 1999.
  39. P. Sereda, A. Vilanova, and F. A. Gerritsen. "Automating transfer function design for volume rendering using hierarchical clustering of material boundaries." In EuroVis, pp. 243-250, 2006.
  40. T. Pfaffelmoser, M. Reitinger, and R. Westermann. "Visualizing the positional and geometrical variability of isosurfaces in uncertain scalar fields." Computer Graphics Forum, 30(3):951-960, 2011. https://doi.org/10.1111/j.1467-8659.2011.01944.x
  41. R. Bramon, M. Ruiz, A. Bardera, I. Boada, M. Feixas, and M. Sbert. "Information theory-based automatic multimodal transfer function design." IEEE journal of biomedical and health informatics, 17(4):870-880, 2013. https://doi.org/10.1109/JBHI.2013.2263227
  42. B. Ma and A. Entezari. "Volumetric feature-based classification and visibility analysis for transfer function design." IEEE Transactions on Visualization and Computer Graphics, 24(12):3253-3267, 2018. https://doi.org/10.1109/tvcg.2017.2776935
  43. Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli. "Image quality assessment: from error visibility to structural similarity." IEEE Transactions on Image Processing, 13(4):600-612, 2004. https://doi.org/10.1109/TIP.2003.819861