Journal of the Institute of Electronics and Information Engineers (전자공학회논문지)

The Institute of Electronics and Information Engineers (대한전자공학회)
권호 표지
DOI QR코드

DOI QR Code

A Real-Time Hardware Architecture for Image Rectification Using Floating Point Processing

부동 소수점 연산을 이용한 실시간 영상 편위교정 FPGA 하드웨어 구조 설계

  • Han, Dongil (Dept. of Computer Engineering, Sejong University) ;
  • Choi, Jeahoon (Dept. of Computer Engineering, Sejong University) ;
  • Shin, Ho Chul (Robot/Cognitive Convergence Research Depart ment, ETRI)
  • 한동일 (세종대학교 컴퓨터공학과) ;
  • 최재훈 (세종대학교 컴퓨터공학과) ;
  • 신호철 (한국전자통신연구원 지능형인지기술연구부)
  • Received : 2013.09.12
  • Published : 2014.02.25
Download PDF
⟨ Previous Next ⟩

Abstract

This paper suggests a novel hardware architecture of a real-time rectification which is to remove vertical parallax of an image occurred in the pre-processing stage of stereo matching. As an off-line step, Matlab Toolbox which was designed by J.Y Bouguet, was used to calculate calibration parameter of the image. Then, based on the Heikkila and Silven's algorithm, rectification hardware was designed. At this point, to enhance the precision of the rectified image, floating-point unit was generated by using Xilinx Core Generator. And, we confirmed that proposed hardware design had higher precision compared to other designs while having the ability to do rectification in real-time.

본 논문은 두 대의 카메라로 찍은 영상을 이용하여 사물의 3D 정보를 계산하는 스테레오 매칭(Stereo Matching) 기법의 전처리 과정에 관한 연구이다. 본 논문에서는 카메라 내부의 왜곡 및 두 카메라간의 정렬 문제로 인해 생긴 영상의 수직시차(vertical parallax)를 제거하기 위한 실시간 편위교정(Rectification) 하드웨어 설계 구조를 제안한다. 이를 위한 사전 단계로 J.Y Bouguet이 설계한 Matlab 툴박스를 이용해 영상의 보정 매개변수(calibration parameter)를 구한 후 Heikkila 와 Silven의 알고리즘을 기반으로 하여 편위교정 하드웨어를 설계하였다. 이때 결과 이미지의 정밀도를 높이기 위하여 Xilinx의 Core 생성기를 이용해 부동소수점 연산기를 생성하여 사용하였으며, 이를 통하여 룩업 테이블(Look-Up Table) 등을 사용하여 설계된 타 편위교정 하드웨어에 비해 높은 정밀도를 가지면서도 실시간으로 작동하는 편위교정 하드웨어를 설계할 수 있음을 확인하였다.

Keywords

References

  1. M. Bleyer, M. Gelautz, "Simple but effective tree structure for dynamic programming based stereo matching", VISAPP, pp. 415-422. Madeira, Portugal, Jan 2008.
  2. Q. Yang, L. Wang, R. Yang, S. Wang, M. Liao, and D. Nister. "Real-time Global stereo matching using hierarchical belief propagation", BMVC, vol. 6, pp. 989-998, Sept 2006.
  3. X. Mei, X. Sun, M. Zhou, S. Jiao, H. Wang, and X. Zhang. "On building an accurate stereo matching system on graphics hardware", ICCV Workshops, pp.467-474, Barcelona, Spain, Nov 2011.
  4. D. Papadimitriou and T. Dennis, "Epipolar line estimation and rectification for stereo image pairs", IEEE Transactions on Image Processing, Vol. 5, no. 4, pp. 672-676, Apr 1996. https://doi.org/10.1109/83.491345
  5. Zhang, Zhengyou. "A flexible new technique for camera calibration." IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 22, no. 11, pp. 1330-1334. Nov 2000. https://doi.org/10.1109/34.888718
  6. Lee, Tung-Ying, et al. "Real-time correction of wide-angle lens distortion for images with GPU computing." APCCAS, pp. 456-459, Kaohsiung, Taiwan Dec 2012.
  7. Pantilie, Cosmin D., et al. "Real-Time Image Rectification and Stereo Reconstruction System on the GPU." 10th International Symposium on Parallel and Distributed Computing (ISPDC), 2011 pp. 79-85, Cluj Napoca, Romania, July 2011.
  8. Vancea, Cristian, and Sergiu Nedevschi. "LUT-based image rectification module implemented in FPGA." IEEE International Conference on Intelligent Computer Communication and Processing, pp. 147-154, Cluj-Napoca, Romanina, Sept 2007.
  9. Jia, Yunde, et al. "A miniature stereo vision machine (MSVM-III) for dense disparity mapping." Proceedings of the 17th International Conference on Pattern Recognition, ICPR. Vol. 1, pp. 728-731, Cambridge, UK, Aug 2004.
  10. Yang, Ruigang, and Marc Pollefeys. "A versatile stereo implementation on commodity graphics hardware." Real-Time Imaging, Vol. 11 no. 1, pp. 7-18. Feb 2005. https://doi.org/10.1016/j.rti.2005.04.002
  11. Maldeniya, Buddhika, et al. "Computationally efficient implementation of video rectification in an FPGA for stereo vision applications." 5th International Conference on Information and Automation for Sustainability (ICIAFs), 2010 pp. 129-224, Columbo, Srilanka, Dec 2010.
  12. Villalpando, Carlos Y., et al. "FPGA implementation of stereo disparity with high throughput for mobility applications." Aerospace Conference, 2011 IEEE, pp. 1-10, Big sky, USA, Mar 2011.
  13. Jawed, Khurram, et al. "Real time rectification for stereo correspondence." International Conference on Computational Science and Engineering, CSE'09. Vol. 2, pp. 277-284, Vancouver, Canada, Aug 2009.
  14. Park, Deuk Hyun, et al. "Real time rectification using differentially encoded lookup table." ICUIMC '11 Proceedings of the 5th International Conference on Ubiquitous Information Management and Communication, pp.47, Seoul, Korea, ,Feb 2011.
  15. Pohl, Matthias, et al. "An efficient and scalable architecture for real-time distortion removal and rectification of live camera images." International Conference on Reconfigurable Computing and FPGAs (ReConFig), pp. 1-7, Cancun, Maxico ,Dec 2012.
  16. Rodrigues, Joao GP, and Joao Canas Ferreira. "FPGA-based rectification of stereo images." Conference on Design and Architectures for Signal and Image Processing (DASIP), 2010 pp. 199-206, Edinnburgh, UK, Oct 2010.
  17. Zicari, Paolo. "Efficient and high performance FPGA-based rectification architecture for stereo vision." Microprocessors and Microsystems, Vol. 37, No. 8, Nov 2013.
  18. Islam, Jamin, et al. "Lowering power consumption using run-time reconfiguration for stereo rectification." Canadian Conference on Electrical and Computer Engineering, 2008. CCECE 2008. pp. 001693 - 001698, Niagara Falls, Canada, May 2008.
  19. Hyo Su Jeong, "Real-time stereo image matching system", Thesis for master's degree of Kyungpook National University, 2011.
  20. Huiyan, H. A. N., H. A. N. Xie, and Yang Fengbao. "Rectification of Uncalibrated Images for Stereo Vision." TELKOMNIKA Indonesian Journal of Electrical Engineering Vol.11, No. 1, pp. 322-329, Jan 2013.
  21. Fusiello, Andrea, and Luca Irsara. "Quasi-Euclidean epipolar rectification of uncalibrated images." Machine Vision and Applications, Vol. 22, No. 4, pp. 663-670, July 2011. https://doi.org/10.1007/s00138-010-0270-3
  22. online : http://www.xilinx.com/tools/ coregen.htm
  23. online : http://www.vision.caltech.edu/bouguetj/calib_doc
  24. Heikkila, J. and Silven, O., "A four-step camera calibration procedure with implicit image correction," EEE Computer Society Conference on Computer Vision and Pattern Recognition, 1997. Proceedings., 1997 Ipp. 1106-1112, San Juan, Puerto Rico, Jun 1997.
  25. Gremban, Keith D., Charles E. Thorpe, and Takeo Kanade. "Geometric camera calibration using systems of linear equations." EEE International Conference on Robotics and Automation, 1988. Proceedings, 1988 I pp. 562-567, Philadelphia, USA, Apr 1988.
  26. online : http://www.vision.caltech.edu/bouguetj/calib_doc/htmls/parameters.html
  27. Minxi Jin, Tsutomu Maruyama, "A Real-time Stereo Vision System Using a Tree-structured Dynamic Programming on FPGA", FPGA '12 Proceedings of the ACM/SIGDA international symposium on Field Programmable Gate Arrays, pp.21-24, Monterey, USA 2012.