Browse > Article
http://dx.doi.org/10.6109/jicce.2014.12.2.089

Three-Dimensional Imaging and Display through Integral Photography  

Navarro, Hector (3D Imaging and Display Lab., Department of Optics, University of Valencia)
Dorado, Adrian (3D Imaging and Display Lab., Department of Optics, University of Valencia)
Saavedra, Genaro (3D Imaging and Display Lab., Department of Optics, University of Valencia)
Corral, Manuel Martinez (3D Imaging and Display Lab., Department of Optics, University of Valencia)
Publication Information
Journal of information and communication convergence engineering / v.12, no.2, 2014 , pp. 89-96 More about this Journal
Abstract
Here, we present a review of the proposals and advances in the field of three-dimensional (3D) imaging acquisition and display made in the last century. The most popular techniques are based on the concept of stereoscopy. However, stereoscopy does not provide real 3D experience, and produces discomfort due to the conflict between convergence and accommodation. For this reason, we focus this paper on integral imaging, which is a technique that permits the codification of 3D information in an array of 2D images obtained from different perspectives. When this array of elemental images is placed in front of an array of microlenses, the perspectives are integrated producing 3D images with full parallax and free of the convergence-accommodation conflict. In the paper we describe the principles of this technique, together with some new applications of integral imaging.
Keywords
Computational photography; Integral imaging; Plenoptic imaging; Three-dimensional display;
Citations & Related Records
연도 인용수 순위
  • Reference
1 J. Arai, M. Okui, M. Kobayashi, and F. Okano, "Geometrical effects of positional errors in integral photography," Journal of the Optical Society of America A, vol. 21, no. 6, pp. 951-958, 2004.
2 M. S. Min, J. Hong, and B. Lee, "Analysis of an optical depth converter used in a three-dimensional integral imaging system," Applied Optics, vol. 43, no. 23, pp. 4539-4549, 2004.   DOI
3 J. Arai, H. Kawai, M. Kawakita, and F. Okano, "Depth-control method for integral imaging," Optics Letters, vol. 33, no. 3, pp. 279-281, 2008.   DOI   ScienceOn
4 H. Navarro, R. Martinez-Cuenca, G. Saavedra, M. Martinez-Corral, and B. Javidi, "3D integral imaging display by smart pseudoscopic-to-orthoscopic conversion (SPOC)," Optics Express, vol. 18, no. 25, pp. 25573-25583, 2010.   DOI
5 B. Tavakoli, M. Daneshpanah, B. Javidi, and E. Watson, "Performance of 3D integral imaging with position uncertainty," Optics Express, vol. 15, no. 19, pp. 11889-11902, 2007.   DOI
6 A. Aggoun, "Pre-processing of integral images for 3-D displays," Journal of Display Technology, vol. 2, no. 4, pp. 393-400, 2006.   DOI   ScienceOn
7 N. P. Sgouros, S. S. Athineos, M. S. Sangriotis, P. G. Papageorgas, and N. G. Theofanous, "Accurate lattice extraction in integral images," Optics Express, vol. 14, no. 22, pp. 10403-10409, 2006.   DOI
8 S. Jung, J. H. Park, H. Choi, and B. Lee, "Viewing-angleenhanced integral three-dimensional imaging along all directions without mechanical movement," Optics Express, vol. 11, no. 12, pp. 1346-1356, 2003.   DOI
9 B. Lee, S. Jung, and J. H. Park, "Viewing-angle-enhanced integral imaging by lens switching," Optics Letters, vol. 27, no. 10, pp. 818-820, 2002.   DOI   ScienceOn
10 J. S. Jang and B. Javidi, "Improvement of viewing angle in integral imaging by use of moving lenslet arrays with low fill factor," Applied Optics, vol. 42, no. 11, pp. 1996-2002, 2003.   DOI
11 B. Javidi and F. Okano, Three-Dimensional Television, Video, and Display Technologies. Berlin: Springer, 2002.
12 A. Isaksen, L. McMillan, and S. J. Gortler, "Dynamically reparameterized light fields," in Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques, New Orleans, LA, pp. 297-306, 2000.
13 V. Vaish, G. Garg, E. V. Talvala, E. Antunez, B. Wilburn, M. Horowitz, and M. Levoy, "Synthetic aperture focusing using a shear-warp factorization of the viewing transform," in Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshop, San Diego, CA, pp. 129-129, 2005.
14 R. Ng, M. Levoy, M. Bredif, G. Duval, M. Horowitz, and P. Hanrahan, "Light field photography with a hand-held plenoptic camera," Computer Science Technical Report, vol. 2, pp. 1-11, 2005.
15 A. Lumsdaine and T. Georgiev, "The focused plenoptic camera," in Proceedings of IEEE International Conference on Computational Photography, San Francisco, CA, pp. 1-8, 2009.
16 K. Fife, A. El Gamal, and H. S. P. Wong, "A multi-aperture image sensor with 0.7 ${\mu}$m pixels in 0.11 ${\mu}m$ CMOS technology," IEEE Journal of Solid-State Circuits, vol. 43, no. 12, pp. 2990-3005, 2008.   DOI   ScienceOn
17 J. Arai, F. Okano, H. Hoshino, and I. Yuyama, "Gradient-index lens-array method based on real-time integral photography for three-dimensional images," Applied Optics, vol. 37, no. 11, pp. 2034-2045, 1998.   DOI   ScienceOn
18 N. Davies and M. McCormick, "Imaging system," US Patent 5,615,048, 1997.
19 N. A. Davies, M. Brewin, and M. McCormick, "Design and analysis of an image transfer system using microlens arrays," Optical Engineering, vol. 33, no. 11, pp. 3624-3633, 1994.   DOI   ScienceOn
20 N. Davies, M. McCormick, and L. Yang, "Three-dimensional imaging systems: a new development," Applied Optics, vol. 27, no. 21, pp. 4520-4528, 1988.   DOI   ScienceOn
21 N. Davies and M. McCormick, "Lens system with intermediate optical transmission microlens screen," U.S. Patent 5,650,876, 1997.
22 N. Davies, and M. McCormick, "Imaging arrangements," U.S. Patent No. 5,655,043, 1997.
23 N. Davies and M. McCormick, "Imaging arrangements," U.S. Patent 6,097,541, 2000.
24 E. H. Adelson and J. R. Bergen, "The plenoptic function and the elements of early vision," Computational Models of Visual Processing, vol. 1, no. 2, pp. 3-20, 1991.
25 J. J. Gibson, The Senses Considered as Perceptual Systems. Boston, MA: Houghton Mifflin, 1966.
26 M. Levoy and P. Hanrahan, "Light field rendering," in Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques, New Orleans, LA, pp. 31-42, 1996
27 S. J. Gortler, R. Grzeszczuk, R. Szeliski, and M. F. Cohen, "The lumigraph," in Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques, New Orleans, LA, pp. 43-54, 1996
28 Y. A. Dudnikov, "Elimination of pseudoscopy in integral photography," Optical Technology, vol. 38, no. 3, pp. 140-143, 1971.
29 T. Okoshi, A. Yano, and Y. Fukumori, "Curved triple-mirror screen for projection-type three-dimensional display," Applied Optics, vol. 10, no. 3, pp. 482-489, 1971.   DOI
30 T. Okoshi, Three-Dimensional Imaging Techniques. New York, NY: Academic Press, 1976.
31 Y. A. Dudnikov, "Autostereoscopy and integral photography," Optical Technology, vol. 37, no. 7, pp. 422-426, 1970.
32 Yu. A. Dudnikov, "Effect of three-dimensional moire in integral photography," Soviet Journal of Optical Technology, vol. 41, no. 5, pp. 260-262, 1974.
33 Y. A. Dudnikov and B. K. Rozhkov, "Selecting the parameters of the lens-array photographing system in integral photography," Soviet Journal of Optical Technology, vol. 45, no. 6, pp. 349-351, 1978.
34 Y. A. Dudnikov and B. K. Rozhkov, "Limiting capabilities of photographing various subjects by the integral photography method," Soviet Journal of Optical Technology, vol. 46, no. 12, pp. 736-738, 1979.
35 Y. A. Dudnikov, B. K. Rozhkov, and E. N. Antipova, "Obtaining a portrait of a person by the integral photography method," Soviet Journal of Optical Technology, vol. 47, no. 9, pp. 562-563, 1980.
36 A. Chutjian and R. J. Collier, "Recording and reconstructing three-dimensional images of computer-generated subjects by Lippmann integral photography," Applied Optics, vol. 7, no. 1, pp. 99-103, 1968.   DOI
37 L. Yang, M. McCormick, and N. Davies, "Discussion of the optics of a new 3-D imaging system," Applied Optics, vol. 27, no. 21, pp. 4529-4534, 1988.   DOI
38 R. L. De Montebello, "Process of making reinforced lenticular sheet," U.S. Patent 3,584,369, 1971.
39 D. F. w. Coffey, "Apparatus for making a composite stereograph," U.S. Patent 2,063,985, 1936.
40 J. T. Gruetzner, "Means for obtaining three-dimensional photography," U.S. Patent 2,724,312, 1955.
41 P. J., Bos and K. R. Koehler/Beran, "The pi-cell: a fast liquidcrystal optical-switching device," Molecular Crystals and Liquid Crystals, vol. 113, no. 1, pp. 329-339, 1984.   DOI
42 W. Rollmann, "Notiz zur stereoskopie," Annalen der Physik, vol. 165, no. 6, pp. 350-351, 1853.   DOI
43 E. H. Land, "Polarizing optical system," U.S. Patent 2,099,694, 1937.
44 D. W. Byatt, "Stereoscopic television system," U.S. Patent 4,281,341, 1981.
45 W. Hess, "Stereoscopic picture," U.S. Patent 1,128,979, 1915.
46 A. Berthier, "Images stereoscopiques de grand format," Le Cosmos, no. 34, pp. 205-210, 1896.
47 B. Julesz, "Stereopsis and binocular rivalry of contours," Journal of the Optical Society of America, vol. 53, no. 8, pp. 994-998, 1963.   DOI
48 C. W. Kanolt, "Photographic method and apparatus," U. S. Patent 1,260,682, 1918.
49 H. Imai, M. Imai, Y. Ogura, and K. Kubota, "Eye-position tracking stereoscopic display using image-shifting optics," Proceedings of SPIE, vol. 2653, pp. 49-55, 1996.
50 M. Levoy, Z. Zhang, and I. McDowall, "Recording and controlling the 4D light field in a microscope using microlens arrays," Journal of Microscopy, vol. 235, no. 2, pp. 144-162, 2009.   DOI   ScienceOn
51 C. B. Burckhardt, R. J. Collier, and E. T. Doherty, "Formation and inversion of pseudoscopic images," Applied Optics, vol. 7, no. 4, pp. 627-631, 1968.   DOI   ScienceOn
52 R. L. De Montebello, "Integral photography," U.S. Patent 3,503,315, 1970.
53 R. L. De Montebello, "Wide-angle integral photography: the integram system," Proceedings of the SPIE, vol. 120, pp. 73-91, 1977
54 H. S. Buck, R. L. de Montebello, and R. P. Globus, "Integral photography apparatus and method of forming same," U.S. Patent 4,732,453, 1988.
55 C. B. Burckhardt, "Optimum parameters and resolution limitation of integral photography," Journal of the Optical Society of America, vol. 58, no. 1, pp. 71-74, 1968.   DOI
56 L. P. Dudley, "Integral photography," U.S. Patent 3,613,539, 1971.
57 L. P. Dudley, "Methods of integral photography," U.S. Patent 3,675,553, 1972.
58 T. Okoshi, "Optimum design and depth resolution of lens-sheet and projection-type three-dimensional displays," Applied Optics, vol. 10, no. 10, pp. 2284-2291, 1971.   DOI   ScienceOn
59 H. Navarro, M. Martinez-Corral, B. Javidi, E. Sanchez-Ortiga, A. Doblas, and G. Saavedra, "Axial segmentation of 3D images through syntetic-apodization integral-imaging microscopy," in Proceedings of Focus on Microscopy Conference, Konstanz, Germany, 2011.
60 D. Shin, M. Cho, and B. Javidi, "Three-dimensional optical microscopy using axially distributed image sensing," Optics Letters, vol. 35, no. 21, pp. 3646-3648, 2010.   DOI   ScienceOn
61 I. Moon and B. Javidi, "Three-dimensional visualization of objects in scattering medium by use of computational integral imaging," Optics Express, vol. 16, no. 17, pp. 13080-13089, 2008.   DOI
62 I. Moon and B. Javidi, "Three-dimensional recognition of photonstarved events using computational integral imaging and statistical sampling," Optics Letters, vol. 34, no. 6, pp. 731-733, 2009.   DOI   ScienceOn
63 D. Aloni, A. Stern, and B. Javidi, "Three-dimensional photon counting integral imaging reconstruction using penalized maximum likelihood expectation maximization," Optics Express, vol. 19, no. 20, pp. 19681-19687, 2011.   DOI
64 S. H. Hong and B. Javidi, "Three-dimensional visualization of partially occluded objects using integral imaging," Journal of Display Technology, vol. 1, no. 2, pp. 354-359, 2005.   DOI   ScienceOn
65 J. S. Jang and B. Javidi, "Three-dimensional integral imaging of micro-objects," Optics Letters, vol. 29, no. 11, pp. 1230-1232, 2004.   DOI   ScienceOn
66 M. Levoy, R. Ng, A. Adams, M. Footer, and M. Horowitz, "Light field microscopy," ACM Transactions on Graphics, vol. 25, no. 3, pp. 924-934, 2006.   DOI   ScienceOn
67 D. Gabor, D. "A new microscopic principle," Nature, vol. 161, no. 4098, pp. 777-778, 1948.   DOI
68 B. Javidi, I. Moon, and S. Yeom, "Three-dimensional identification of biological microorganism using integral imaging," Optics Express, vol. 14, no. 25, pp. 12096-12108, 2006.   DOI
69 G. Lippmann, "Epreuves reversibles donnant la sensation du relief," Journal de Physique Theorique et Appliquee, vol. 7, no. 1, pp. 821-825, 1908.   DOI
70 B. G. Blundell and A. J. Schwarz, Volumetric Three-Dimensional Display Systems. New York, NY: John Wiley & Sons, 2000.
71 G. Lippmann, "Epreuves reversibles photographies integrals," Comptes Rendus de l'Academie des Sciences, vol. 146, pp. 446-451, 1908.
72 G. Lippmann, "L'etalon international de radium," Radium (Paris), vol. 9, no. 4, pp. 169-170, 1912.   DOI
73 Y. Kim et al., "Accommodative response of integral imaging in near distance," Journal of Display Technology, vol. 8, no. 2, pp. 70-78, 2012.   DOI   ScienceOn
74 H. Hiura, S. Yano, T. Mishina, J. Arai, K. Hisatomi, Y. Iwadate, and T. Ito, "A study on accommodation response and depth perception in viewing integral photography," in Proceedings of the 5th International Conference on 3D Systems and Applications (3DSA), Osaka, Japan, 2013.
75 M. Martinez-Corral, R. Martinez-Cuenca, G. Saavedra, H. Navarro, A. Pons, and B. Javidi, "Progresses in 3D integral imaging with optical processing," Journal of Physics: Conference Series, vol. 139, no. 1, p. 012012, 2008.   DOI   ScienceOn
76 H. E. Ives, "Optical properties of a Lippman lenticulated sheet," Journal of the Optical Society of America, vol. 21, no. 3, pp. 171-176, 1931.   DOI
77 O. Matoba, E. Tajahuerce, and B. Javidi, "Real-time threedimensional object recognition with multiple perspectives imaging," Applied Optics, vol. 40, no. 20, pp. 3318-3325, 2001.   DOI   ScienceOn
78 H. E. Ives, "Parallax panoramagrams made with a large diameter lens," Journal of the Optical Society of America, vol. 20, no. 6, pp. 332-340, 1930.   DOI
79 S. H. Hong and B. Javidi, "Distortion-tolerant 3D recognition of occluded objects using computational integral imaging," Optics Express, vol. 14, no. 25, pp. 12085-12095, 2006.   DOI
80 J. Arai, F. Okano, M. Kawakita, M. Okui, Y. Haino, M. Yoshimura,... and M. Sato, "Integral three-dimensional television using a 33-megapixel imaging system," Journal of Display Technology, vol. 6, no. 10, pp. 422-430, 2010.   DOI   ScienceOn
81 S. Kishk and B. Javidi, "Improved resolution 3D object sensing and recognition using time multiplexed computational integral imaging," Optics Express, vol. 11, no. 26, pp. 3528-3541, 2003.   DOI
82 R. Schulein, C. M. Do, and B. Javidi, "Distortion-tolerant 3D recognition of underwater objects using neural networks," Journal of the Optical Society of America A, vol. 27, no. 3, pp. 461-468, 2010.   DOI   ScienceOn
83 S. Yeom, B. Javidi, and E. Watson, "Three-dimensional distortiontolerant object recognition using photon-counting integral imaging," Optics Express, vol. 15, no. 4, pp. 1513-1533, 2007.   DOI
84 B. Tavakoli, B. Javidi, and E. Watson, "Three dimensional visualization by photon counting computational integral imaging," Optics Express, vol. 16, no. 7, pp. 4426-4436, 2008.   DOI
85 M. Martinez-Corral, B. Javidi, R. Martinez-Cuenca, and G. Saavedra, "Integral imaging with improved depth of field by use of amplitude-modulated microlens arrays," Applied Optics, vol. 43, no. 31, pp. 5806-5813, 2004.   DOI
86 M. DaneshPanah, B. Javidi, and E. Watson, "Three dimensional object recognition with photon counting imagery in the presence of noise," Optics Express, vol. 18, no. 25, pp. 26450-26460, 2010.   DOI
87 A. Castro, Y. Frauel, and B. Javidi, "Integral imaging with large depth of field using an asymmetric phase mask," Optics Express, vol. 15, no. 16, pp. 10266-10273, 2007.   DOI
88 R. Martinez-Cuenca, H. Navarro, G. Saavedra, B. Javidi, and M. Martinez-Corral, "Enhanced viewing-angle integral imaging by multiple-axis telecentric relay system," Optics Express, vol. 15, no. 24, pp. 16255-16260, 2007.   DOI
89 R. Martinez-Cuenca, G. Saavedra, M. Martinez-Corral, and B. Javidi, "Extended depth-of-field 3-D display and visualization by combination of amplitude-modulated microlenses and deconvolution tools," Journal of Display Technology, vol. 1, no. 2, pp. 321-327, 2005.   DOI   ScienceOn
90 R. Martinez-Cuenca, G. Saavedra, A. Pons, B. Javidi, and M. Martinez-Corral, "Facet braiding: a fundamental problem in integral imaging," Optics Letters, vol. 32, no. 9, pp. 1078-1080, 2007.   DOI   ScienceOn
91 F. Okano, J. Arai, K. Mitani, and M. Okui, "Real-time integral imaging based on extremely high resolution video system," Proceedings of the IEEE, vol. 94, no. 3, pp. 490-501, 2006.   DOI   ScienceOn
92 T. Mishina, "3D television system based on integral photography," in Proceedings of the 28th Picture Coding Symposium, Nagoya, Japan, pp. 20-20, 2010.
93 N. Davies and M. McCormick, "Imaging system," US Patent 5,040,871, 1991.
94 A. P. Sokolov, Autostereoscopy and integral photography by Professor Lippmann's method. Moskva: Izd-vo MGU (Moscow State University), 1911.
95 F. E. Ives, "A novel stereogram," Journal of the Franklin Institute, vol. 153, no. 1, pp. 51-52, 1902.   DOI   ScienceOn
96 E. H. Adelson and J. Y A. Wang, "Single lens stereo with a plenoptic camera," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 14, no. 2, pp. 99-106, 1992.   DOI   ScienceOn
97 C. B. Burckhardt and E. T. Doherty, "Beaded plate recording of integral photographs," Applied Optics, vol. 8, no. 11, pp. 2329-2331, 1969.   DOI
98 H. Navarro, R. Martinez-Cuenca, A. Molina-Martin, M. Martinez-Corral, G. Saavedra, and B. Javidi, "Method to remedy image degradations due to facet braiding in 3D integral-imaging monitors," Journal of Display Technology, vol. 6, no. 10, pp. 404-411, 2010.   DOI   ScienceOn
99 F. Okano, H. Hoshino, J. Arai, and I. Yuyama, "Real-time pickup method for a three-dimensional image based on integral photography," Applied Optics, vol. 36, no. 7, pp. 1598-1603, 1997.   DOI   ScienceOn