Journal of the Institute of Electronics and Information Engineers (전자공학회논문지)

The Institute of Electronics and Information Engineers (대한전자공학회)
권호 표지
DOI QR코드

DOI QR Code

Image Cache for FPGA-based Real-time Image Warping

FPGA 기반 실시간 영상 워핑을 위한 영상 캐시

  • Choi, Yong Joon (Dept. of Electrical and Information Eng., Seoul National University of Science and Technology) ;
  • Ryoo, Jung Rae (Dept. of Electrical and Information Eng., Seoul National University of Science and Technology)
  • 최용준 (서울과학기술대학교 전기정보공학과) ;
  • 류정래 (서울과학기술대학교 전기정보공학과)
  • Received : 2016.03.01
  • Accepted : 2016.05.28
  • Published : 2016.06.25
Download PDF
⟨ Previous Next ⟩

Abstract

In FPGA-based real-time image warping systems, image caches are utilized for fast readout of image pixel data and reduction of memory access rate. However, a cache algorithm for a general computer system is not suitable for real-time performance because of time delays from cache misses and on-line computation complexity. In this paper, a simple image cache algorithm is presented for a FPGA-based real-time image warping system. Considering that pixel data access sequence is determined from the 2D coordinate transformation and repeated identically at every image frame, a cache load sequence is off-line programmed to guarantee no cache miss condition, and reduced on-line computation results in a simple cache controller. An overall system structure using a FPGA is presented, and experimental results are provided to show accuracy and validity of the proposed cache algorithm.

FPGA 기반 실시간 영상 워핑 시스템에서는 영상 픽셀 정보의 빠른 읽기와 메모리 접근 횟수의 감소를 위하여 영상 캐시를 활용하지만, 일반 컴퓨터 시스템의 캐시 알고리즘은 캐시 부적중(cache miss)에 의한 시간 지연과 복잡한 온라인(on-line) 연산 구조로 인하여 실시간 성능 구현에 어려움이 있다. 본 논문에서는 FPGA 기반 실시간 영상 워핑을 위한 단순한 구조의 영상 캐시 알고리즘을 제안한다. 영상 워핑에서의 픽셀 데이터 접근 순서는 워핑에 적용할 2D 좌표변환 관계에 의하여 결정되며 매 영상 프레임에서 반복되는 특성이 있다. 따라서, 캐시 로드(cache load)에 관한 사항을 오프라인(off-line)에서 미리 프로그램함으로써 캐시 부적중 상황이 발생하지 않음을 보장할 수 있고, 그 결과 온라인에서의 연산이 감소하여 캐시 컨트롤러의 구조가 단순해진다. FPGA를 활용한 전체 시스템 구조를 제시하고, 실험을 통하여 제안하는 영상 캐시 알고리즘의 정확성과 타당성을 확인한다.

Keywords

References

  1. J. Park, S.-C. Byun, and B.-U. Lee, "Lens distortion correction using ideal image coordinates," IEEE Transactions on Consumer Electronics, vol. 55, no. 3, pp. 987-991, Aug. 2009. https://doi.org/10.1109/TCE.2009.5278053
  2. Z. Chen, C. Wu, and H. T. Tsui, "A new image rectification algorithm," Pattern Recognition Letters, vol. 24, no. 1-3, pp. 251-260, Jan. 2003. https://doi.org/10.1016/S0167-8655(02)00239-8
  3. M. Brown and D. G. Lowe, "Automatic panoramic image stitching using invariant features," International Journal of Computer Vision, vol. 74, no. 1, pp. 59-73, Aug. 2007. https://doi.org/10.1007/s11263-006-0002-3
  4. R. Melo, G. Falcao, and J. P. Barreto, "Real-time HD image distortion correction in heterogenous parallel computing systems using efficient memory access patterns," Journal of Real-Time Image Processing, vol. 11, no. 1, pp. 83-91, Jan. 2016. https://doi.org/10.1007/s11554-012-0304-3
  5. J. Park, J. Choi, B.-K. Seo, and J.-I. Park, "Fast stereo image rectification using mobile GPU," in Proc. of the 3rd International Conference on Digital Information Processing and Communications 2013, pp. 485-488, Dubai, UAE, Jan. 2013.
  6. E. Peek, B. Wunsche, and C. Lutterroth, "Using integrated GPUs to perform image warping for HMD," in Proc. of the 29th International Conference on Image and Vision Computing New Zealand, pp. 172-177, Hamilton, New Zealand, Nov. 2014.
  7. W. Wang, J. Yan, N. Xu, Y. Wang, and F.-H. Hsu, "Real-time high-quality stereo vision system in FPGA," IEEE Transactions on Circuits and Systems for Video Technology, vol. 25, no. 10, pp. 1691-1708, Oct. 2015.
  8. H. M. Moon and S. B. Pan, "VLSI architecture of digital image scaler combining linear interpolation and cubic convolution interpolation," Journal of The Institute of Electronics and Information Engineers, vol. 51, no. 3, pp. 579-584, March 2014.
  9. D. Han, J. Choi, and H. C. Shin, "A real-time hardware architecture for image rectification using floating point processing," Journal of The Institute of Electronics and Information Engineers, vol. 51, no. 2, pp. 342-353, Feb. 2014.
  10. J. R. Ryoo, E. S. Lee, and T.-Y. Doh, "An implementation of real-time image warping using FPGA," IEMEK Journal of Embedded Systems and Applications, vol. 9, no. 6, pp. 335-344, 2014. https://doi.org/10.14372/IEMEK.2014.9.6.335
  11. P. Greisen, S. Heinzle, M. Gross, and A. P Burg, "An FPGA-based processing pipeline for high-definition stereo video," Journal of Image and Video Processing, vol. 2011, no. 18, pp. 1-13, Nov. 2011.
  12. J.-h Kim, J.-g. Kim, J.-k. Oh, S.-m. Kang, and J.-D. Cho, "Efficient hardware implementation of real-time rectification using adaptively compressed LUT," Journal of Semiconductor Technology and Science, vol. 16, no. 1, pp. 44-57, Feb. 2016. https://doi.org/10.5573/JSTS.2016.16.1.044
  13. J. Moon and J. Kim, "Real-time FPGA rectification implementation combined with stereo camera," in Proc. of the 2015 IEEE International Symposium on Consumer Electronics, pp. 1-2, Madrid, Spain, June 2015.
  14. P. Zicari, "Efficient and high performance FPGA-based rectification architecture for stereo vision," Microprocessors and Microsystems, vol. 37, no. 8, pp. 1144-1154, Nov. 2013. https://doi.org/10.1016/j.micpro.2013.09.007
  15. X. Yuan, Z. Qinghai, G. Liwei, Z. Mingcheng, and R. K. F. Teng, "Study of a FPGA real-time multi-cameras cylindrical panorama video system with low latency and high performance," in Proc. of the 11th Image, Video and Multidimensional Signal Processing Workshop, pp. 1-4, Seoul, Korea, June 2014.
  16. J. L. Hennessy and D. A. Patterson, Computer Architecture a Quantitative Approach, Morgan Kaufmann Publishers, pp. 375-427, 1996.