Real-time Disparity Acquisition Algorithm from Stereoscopic Image and its Hardware Implementation

스테레오 영상으로부터의 실시간 변이정보 획득 알고리듬 및 하드웨어 구현

  • 신완수 (광운대학교 실감미디어 연구소) ;
  • 최현준 (광운대학교 실감미디어 연구소) ;
  • 서영호 (광운대학교 실감미디어 연구소) ;
  • 김동욱 (광운대학교 실감미디어 연구소)
  • Published : 2009.11.30

Abstract

In this paper, the existing disparity aquisition algorithms were analyzed, on the bases of which a disparity generation technique that is superior in accuracy to the generation time was proposed. Basically it uses a pixel-by-pixel motion estimation technique. It has a merit of possibility of a high-speed operation. But the motion estimation technique has a disadvantage of lower accuracy because it depends on the similarity of the matching window regardless of the distribution characteristics of the texture in an image. Therefore, an enhanced technique to increase the accuracy of the disparity is required. This paper introduced a variable-sized window matching technique for this requirement. By the proposed technique, high accuracies could be obtained at the homogeneous regions and the object edges. A hardware to generate disparity image was designed, which was optimized to the processing speed so that a high throughput is possible. The hardware was designed by Verilog-HDL and synthesized using Hynix $0.35{\mu}m$ CMOS cell library. The designed hardware was operated stably at 120MHz using Cadence NC-VerilogTM and could process 15 frames per second at this clock frequency.

본 논문에서는 기존의 변이 영상 획득 방법들에 비하여 시간 대비 정확도가 우수한 기법을 제안하고 H/W로 구현한다. 제안한 기법은 고속 연산이 가능한 화소 대 화소의 움직임 추정 기법을 이용한다. 움직임 추정 기법은 영상 내 텍스쳐의 분포 특성과 무관하게 정합 윈도우의 유사성에만 의존하기 때문에 추출된 변이정보의 정확도가 떨어진다. 이를 해결하기 위해서 영상의 국부 특성에 따른 가변 크기 윈도우 정합 기법을 도입하고, 영상 내 텍스쳐가 균일한 부분 및 물체의 윤곽선 부분에서도 높은 정확도를 얻는다. 제안한 기법은 고속 연산이 가능하도록 수행속도에 최적화된 하드웨어로 설계된다. 하드웨어는 Verilog-HDL로 설계하였고, Hynix $0.35{\mu}m$ CMOS 라이브러리를 사용하여 게이트수준으로 합성하였다. 구현한 하드웨어는 최대 120MHz의 클록 주파수에서 초당 15 프레임을 안정적으로 처리할 수 있었다.

Keywords

References

  1. ISO/IEC MPEG & ITU-T VCEG, 'Multiview video plus depth (MVD) format for advanced 3D video systems,' JVT-W100, April 2007
  2. ISO/IEC, 'ISO/IEC JTC1/SC29/WG11 Coding of Moving Picture and Audio,' Draft of version 4 of ISO/IEC 14496-10 (E) MPEG05/N7081, April 2005
  3. D. Scharstein and R. Szeliski, 'A taxonomy and evaluation of dense two-frame stereo correspondence algorithms,' International Journal of Computer Vision, Vol. 47, Issue 1-3, pp. 7-42, April 2002 https://doi.org/10.1023/A:1014573219977
  4. J. Sun, N. N. Zheng, and H. Y. Shum, 'Stereo matching using belief propagation,' IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 27, Issue 25, pp. 787-800, July 2003
  5. P. N. Belhumeur, 'A Bayesian approach to binocular stereopsis,' International Journal of Computer Vision, Vol. 19, Issue 3, pp. 237-260, Aug. 1996 https://doi.org/10.1007/BF00055146
  6. I. Gallo, E. Binaghi, and M. Raspanti, 'Neural disparity computation for dense two-frame stereo correspondence,' Pattern Recognition Letters, Vol. 29, Issue 5, pp. 673-687, April 2008 https://doi.org/10.1016/j.patrec.2007.12.003
  7. Y. Ohta and T. Kanade, 'Stereo by intra- and inter-scanline search using dynamic programming,' IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. PAMI-7, March 1985 https://doi.org/10.1109/TPAMI.1985.4767639
  8. C. J. Tsa and A. K. Katsaggelos, 'Dense disparity estimation with a divide-and-conquer disparity space image technique,' IEEE Transactions on Multimedia, Vol. 1, Issue 1, pp. 18-29, March 1999 https://doi.org/10.1109/6046.748168
  9. C. Georgoulas, L. Kotoulas, G. Ch. Sirakoulis, I. Andreadis, and A. Gasteratos, 'Real-time disparity map computation module,' Microprocessors and Microsystems, Vol. 32, Issue 3, pp. 159-170, May 2008 https://doi.org/10.1016/j.micpro.2007.10.002
  10. D. I. Han, B. M. Lee, J. I. Cho, and D. H. Hwang, 'Real-time object segmentation using disparity map of stereo matching,' Applied Mathematics and Computation, Vol. 205, Issue 2, pp. 770-777, Nov. 2008 https://doi.org/10.1016/j.amc.2008.05.110
  11. M. Z. Brown, D. Burschka, and G. D. Hager, 'Advances in computational stereo,' IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 25, Issue 8, pp. 993-1008 Aug. 2003 https://doi.org/10.1109/TPAMI.2003.1217603
  12. ISO/IEC 11172-2, 'Information technology coding of moving picture and associated audio for digital storage media at up to about 1.5Mbps,' International Standard, 1993
  13. ISO/IEC 11172-2, 'Information technology coding of moving picture and associated : video,' International Standard, 1995
  14. ISO/IEC 14496-2, 'Information technology coding of audio-visual object,' International Standard, 2001
  15. R. C. Gonzales and R. E. Woods, 'Digital image processing,' Prentice Hall, 2nd edtion, 2001
  16. http://vision.middlebury.edu/stereo
  17. Y. Wang, J. Ostermann, and Y. Q. Zhang, 'Video Processing and Communications,' Prentice hall, 2002
  18. H. Niitsuma, and T. Maruyama, 'Real-time detection of moving object,' Lecture Notes in Computer Science, Vol. 3203, pp.1155-1157, 2004 https://doi.org/10.1007/978-3-540-30117-2_154