Browse > Article
http://dx.doi.org/10.15701/kcgs.2019.25.2.31

Gadget Arms: Interactive Data Visualization using Hand Gesture in Extended Reality  

Choi, JunYoung (Ulsan National Institute of Science and Technology (UNIST))
Jeong, HaeJin (Ulsan National Institute of Science and Technology (UNIST))
Jeong, Won-Ki (Ulsan National Institute of Science and Technology (UNIST))
Publication Information
Journal of the Korea Computer Graphics Society / v.25, no.2, 2019 , pp. 31-41 More about this Journal
Abstract
Extended Reality (XR), such as virtual and augmented reality, has huge potential for immersive data visualization and analysis. In XR, users can interact with data and other users realistically by navigating the shared virtual space, allowing for more intuitive data analysis. However, creating a visualization in XR also poses a challenge because complicated, low-level programming is required, which hinders broad adaptation in visual analytics. This paper proposes an interactive visualization authoring tool based on hand gesture for immersive data visualization-Gadget Arms. The proposed system provides a novel user interaction to create and place visualization in the 3D virtual world. This simple, but intuitive, user interaction enables user designs the entire visualization space in the XR without using a host computer and low-level programming. Our user study also confirmed that the proposed user interaction significantly improves the usability of the visualization authoring tool.
Keywords
Extended Reality; Immersive Data Visualization; Interaction; Hand Gesture; Visual Analytics;
Citations & Related Records
연도 인용수 순위
  • Reference
1 Maxime Cordeil, Tim Dwyer, Karsten Klein, Bireswar Laha, Kim Marriott, and Bruce H Thomas. Immersive collaborative analysis of network connectivity: Cave-style or head-mounted display? IEEE transactions on visualization and computer graphics 23, 1 (2017), 441-450. 2017.   DOI
2 Ciro Donalek, S. George Djorgovski, Scott Davidoff, Alex Cioc, Anwell Wang, Giuseppe Longo, Jeffrey S. Norris, Jerry Zhang, Elizabeth Lawler, Stacy Yeh, Ashish Mahabal, Matthew J. Graham, and Andrew J. Drake. Immersive and collaborative data visualization using virtual reality platforms. 2014 IEEE International Conference on Big Data (Big Data), 609-614. 2014.
3 Adrien Fonnet, Florian Melki, Yannick Prie, Fabien Picarougne, and Gregoire Cliquet. Immersive Data Exploration and Analysis. In Student Interaction Design Research conference. 2018.
4 Joon Hyub Lee, Sang-Gyun An, Yongkwan Kim, and Seok-Hyung Bae. Projective Windows: Bringing Windows in Space to the Fingertip. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, 218. 2018.
5 James R Lewis. IBM computer usability satisfaction questionnaires: psychometric evaluation and instructions for use. International Journal of HumanComputer Interaction 7, 1, 57-78. 1995.   DOI
6 Martin Luboschik, Philip Berger, and Oliver Staadt. On spatial perception issuesin augmented reality based immersive analytics. In Proceedings of the 2016 ACM Companion on Interactive Surfaces and Spaces. ACM, 47-53. 2016.
7 Longfei Ma, Zhe Zhao, Fang Chen, Boyu Zhang, Ligong Fu, and Hongen Liao. Augmented reality surgical navigation with ultrasound-assisted registration for pedicle screw placement: a pilot study. International journal of computer assisted radiology and surgery 12, 12, 2205-2215. 2017.   DOI
8 Patrick Millais, Simon L. Jones, and Ryan Kelly. Exploring Data in Virtual Reality: Comparisons with 2D Data Visualizations. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems (CHI EA '18). ACM, New York, NY, USA, Article LBW007, 6 pages. 2018.
9 Patrick Millais, Simon L Jones, and Ryan Kelly. Exploring Data in Virtual Reality: Comparisons with 2D Data Visualizations. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, LBW007. 2018.
10 Ekaterina Olshannikova, Aleksandr Ometov, Yevgeni Koucheryavy, and Thomas Olsson. Visualizing Big Data with augmented and virtual reality: challenges and research agenda. Journal of Big Data 2, 1 (01 Oct 2015), 22. 2015.   DOI
11 Scott Riddle, Daniel Wasser, and Michael McCarthy. Touching The Human Neuron: User-Centric Augmented Reality Viewing and Interaction of in-vivo Cellular Confocal Laser Scanning Microscopy (CLSM) Utilizing High Resolution zStack Data Sets. Journal of Biocommunication 41, 1, 2017.
12 Ronell Sicat, Jiabao Li, JunYoung Choi, Maxime Cordeil, Won-Ki Jeong, Benjamin Bach, and Hanspeter Pfister. DXR: A Toolkit for Building Immersive Data Visualizations. IEEE Transactions on Visualization and Computer Graphics. 2018.
13 Mauricio Sousa, Daniel Mendes, Soraia Paulo, Nuno Matela, Joaquim Jorge, and Daniel Simoes Lopes. VRRRRoom: Virtual Reality for Radiologists in the Reading Room. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). ACM, New York, NY, USA, 4057-4062. 2017.
14 Daniele Sportillo, Alexis Paljic, Mehdi Boukhris, Philippe Fuchs, Luciano Ojeda, and Vincent Roussarie. An immersive Virtual Reality system for semiautonomous driving simulation: a comparison between realistic and 6-DoF controller-based interaction. In Proceedings of the 9th International Conference on Computer and Automation Engineering. ACM, 6-10. 2017.
15 Bowman, D. A., & Hodges, L. F. An evaluation of techniques for grabbing and manipulating remote objects in immersive virtual environments. In Proceedings of the 1997 symposium on Interactive 3D graphics(pp. 35-ff). ACM. 1997.
16 Will Usher, Pavol Klacansky, Frederick Federer, Peer-Timo Bremer, Aaron Knoll, Jeff Yarch, Alessandra Angelucci, and Valerio Pascucci. A virtual reality visualization tool for neuron tracing. IEEE transactions on visualization and computer graphics 24, 1. 994-1003. 2018.   DOI
17 Eric Whitmire, Hrvoje Benko, Christian Holz, Eyal Ofek, and Mike Sinclair. Haptic Revolver: Touch, Shear, Texture, and Shape Rendering on a Reconfigurable Virtual Reality Controller. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, 86. 2018.
18 Ivan Poupyrev, Mark Billinghurst, Suzanne Weghorst, Tadao Ichikawa: The Go-Go Interaction Technique: Non-Linear Mapping for Direct Manipulation in VR. ACM Symposium on User Interface Software and Technology 79-80, 1996.
19 Google Tilt Brush. https://www.tiltbrush.com/.
20 ARToolkit. www.artoolkit.org;
21 Mixed Reality Toolkit. https://github.com/Microsoft/MixedRealityToolkitUnity
22 D. Filonik, T. Bednarz, M. Rittenbruch, and M. Foth. Glance: Generalized geometric primitives and transformations for information visualization in AR/VR environments. In Proceedings of the 15th ACM SIGGRAPH Conference on Virtual-Reality Continuum and Its Applications in Industry-Volume 1, VRCAI '16, pp. 461-468. ACM, New York, NY, USA, 2016
23 Unity. https://unity3d.com/
24 Inspector Gadget. https://en.wikipedia.org/wiki/Inspector_Gadget.
25 Leap Motion. https://www.leapmotion.com/.
26 Microsoft Docs. https://docs.microsoft.com/en-us/windows/mixed-reality/gestures
27 Hassan Abu Alhaija, Siva Karthik Mustikovela, Lars Mescheder, Andreas Geiger, and Carsten Rother. Augmented Reality Meets Computer Vision: Efficient Data Generation for Urban Driving Scenes. International Journal of Computer Vision 126, 9, 961-972, 2018.   DOI
28 Hrvoje Benko, Christian Holz, Mike Sinclair, and Eyal Ofek. Normaltouch and texturetouch: High-fidelity 3d haptic shape rendering on handheld virtual reality controllers. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology. ACM, 717-728, 2016.
29 Rahul Arora, Rubaiat Habib Kazi, Tovi Grossman, George Fitzmaurice, and Karan Singh. SymbiosisSketch: Combining 2D & 3D Sketching for Designing Detailed 3D Objects in Situ. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, 185, 2018.
30 Benjamin Bach, Ronell Sicat, Hanspeter Pfister, and Aaron Quigley. Drawing into the AR-CANVAS: Designing embedded visualizations for augmented reality. In Workshop on Immersive Analytics, IEEE Vis, 2017.
31 Simon Butscher, Sebastian Hubenschmid, Jens Muller, Johannes Fuchs, and Harald Reiterer. Clusters, Trends, and Outliers: How Immersive Technologies Can Facilitate the Collaborative Analysis of Multidimensional Data. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). ACM, New York, NY, USA, Article 90, 12 pages. 2018.
32 Yeonjoo Cha and Rohae Myung. Extended Fitts' law for 3D pointing tasks using 3D target arrangements. International Journal of Industrial Ergonomics 43, 4, 350-355, 2013.   DOI
33 Inrak Choi, Eyal Ofek, Hrvoje Benko, Mike Sinclair, and Christian Holz. CLAW: A Multifunctional Handheld Haptic Controller for Grasping, Touching, and Triggering in Virtual Reality. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, 654. 2018.
34 Maxime Cordeil, Andrew Cunningham, Tim Dwyer, Bruce H. Thomas, and Kim Marriott. 2017. ImAxes: Immersive Axes As Embodied Affordances for Interactive Multivariate Data Visualisation. In Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology (UIST '17). ACM, New York, NY, USA, 71-83. https://doi.org/10.1145/3126594.3126613