International Journal of Computer Science & Network Security

International Journal of Computer Science & Network Security (국제컴퓨터통신보호논문지학회)
권호 표지
DOI QR코드

DOI QR Code

Multimodal Interaction Framework for Collaborative Augmented Reality in Education

  • Asiri, Dalia Mohammed Eissa (Department of Computer Science, King Abdul-Aziz University) ;
  • Allehaibi, Khalid Hamed (Department of Computer Science, Faculty of Computing and Information Technology, King Abdul-Aziz University) ;
  • Basori, Ahmad Hoirul (Department of Information Technology, Faculty of Computing, and Information Technology in Rabigh, King Abdul-Aziz University)
  • Received : 2022.07.05
  • Published : 2022.07.30
Download PDF
⟨ Previous Next ⟩

Abstract

One of the most important technologies today is augmented reality technology, it allows users to experience the real world using virtual objects that are combined with the real world. This technology is interesting and has become applied in many sectors such as the shopping and medicine, also it has been included in the sector of education. In the field of education, AR technology has become widely used due to its effectiveness. It has many benefits, such as arousing students' interest in learning imaginative concepts that are difficult to understand. On the other hand, studies have proven that collaborative between students increases learning opportunities by exchanging information, and this is known as Collaborative Learning. The use of multimodal creates a distinctive and interesting experience, especially for students, as it increases the interaction of users with the technologies. The research aims at developing collaborative framework for developing achievement of 6th graders through designing a framework that integrated a collaborative framework with a multimodal input "hand-gesture and touch", considering the development of an effective, fun and easy to use framework with a multimodal interaction in AR technology that was applied to reformulate the genetics and traits lesson from the science textbook for the 6th grade, the first semester, the second lesson, in an interactive manner by creating a video based on the science teachers' consultations and a puzzle game in which the game images were inserted. As well, the framework adopted the cooperative between students to solve the questions. The finding showed a significant difference between post-test and pre-test of the experimental group on the mean scores of the science course at the level of remembering, understanding, and applying. Which indicates the success of the framework, in addition to the fact that 43 students preferred to use the framework over traditional education.

Keywords

Acknowledgement

I am indebted to my supervisor, my supervisor, Dr. Khalid Allehaibi, Co-Supervised, Dr. Ahmad Hoirul Basori, thank you for your patience, guidance, and support. I have benefited greatly from your wealth of knowledge and meticulous editing. Not only that, but I am extremely grateful that you took me on as a student and continued to have faith in me over the years. Thank you to the interviewees, who so generously took time out of their schedules to participate in my research and make this project possible. I am grateful for my family's unconditional, unequivocal, and loving support. Thank you to my parents, for your endless support. You have always stood behind me. Thank you to my partner, Eng. Ahmad Al Manea, for constantly listening to me, and for the sacrifices you have made in order for me to pursue a master's degree.

References

  1. M.-B. Ibanez and C. Delgado-Kloos, "Augmented reality for STEM learning: A systematic review," Computers & Education, pp. 109-123, 2018.
  2. D. Nincareana, M. B. Ali, N. D. A. Halim and M. H. A. Rahman, "Mobile Augmented Reality: the potential for education," Procedia-social and behavioral sciences , pp. 657-664, 2013.
  3. M. Billinghurst, "Augmented Reality in Education," New horizons for learning , pp. 1-5, 2002.
  4. B. H. Limbu, H. Jarodzka, R. Klemke and M. Specht, "Using sensors and augmented reality to train apprentices using recorded expert performance: A systematic literature review.," Educational Research Review, pp. 1-22, 2018.
  5. Ali, A. A., G. A. Dafoulas and a. J. C. Augusto., "Collaborative educational environments incorporating mixed reality technologies: A systematic mapping study.," IEEE Transactions on Learning Technologies , pp. 321-332, 2019.
  6. J. Martin-Gutierrez, P. Fabiani, W. Benesova, M. D. Meneses and C. E. M. a, "Augmented reality to promote collaborative and autonomous learning in higher education.," Computers in human behavior , pp. 752-761, 2015. https://doi.org/10.1016/j.chb.2014.11.093
  7. D. N. E. Phon, M. B. Ali and a. N. D. A. Halim., "Collaborative augmented reality in education: A review.," International Conference on Teaching and Learning in Computing and Engineering. IEEE, 2014.
  8. P. e. a. Wang, "A gesture-and head-based multimodal interaction platform for MR remote collaboration.," The International Journal of Advanced Manufacturing Technology, pp. 3031-3043, 2019.
  9. R. Z. H. A. a. S. A. A. S. Abidin, " Adaptive multimodal interaction in mobile augmented reality: A conceptual framework.," AIP Conference Proceedings, 2017.
  10. A. W. B. M. S. M. S. &. Y. C. S. Ismail, " Designing an augmented reality multimodal interface for 6DOF manipulation techniques. ," In Proceedings of SAI Intelligent Systems Conference,Springer, Cham., pp. 309-322, September 2018.
  11. E. a. S. W. Novak, " Effects of 3D printing project-based learning on preservice elementary teachers' science attitudes, science content knowledge, and anxiety about teaching science," Journal of Science Education and Technology, pp. 412-432, 2018.
  12. C. A. K. a. C. S. Troussas, " Collaboration and fuzzy-modeled personalization for mobile game-based learning in higher education.," Computers & Education, 2020.
  13. I. A. T. a. C. L. Kazanidis, "Teaching mobile programming using augmented reality and collaborative game based learning.," Interactive mobile communication, technologies and learning. Springer, Cham, 2017.
  14. H. K. P. Y. L. a. A. S. Alhumaidan, " Co-designing with children a collaborative augmented reality book based on a primary school textbook.," International Journal of Child-Computer Interaction, pp. 24-36, 2018.
  15. P. C. K. a. S. B. Boonbrahm, "Interactive augmented reality: a new approach for collaborative learning.," International conference on learning and collaboration technologies. Springer, Cham, 2016.
  16. P. J. S. P. a. A. G. Sarkar, "ScholAR: a collaborative learning experience for rural schools using Augmented Reality application.," IEEE Tenth International Conference on Technology for Education (T4E). IEEE ., 2018.
  17. J. Barbadillo, N. Barrena, V. Goni and a. J. R. S'anchez, "Collaborative e-learning framework for creating augmented reality mobile educational activities.," International Conference on Ubiquitous Computing and Ambient Intelligence. Springer, Cham ., 2014.
  18. S. Irawati, S. Green and M. Billinghurst, "Move the couch where?: developing an augmented reality multimodal interface.," 2006 IEEE/ACM International Symposium on Mixed and Augmented Reality. IEEE., 2006.
  19. Z. Chen, J. Li, Y. Hua, R. Shen and A. Basu, "Multimodal Interaction in Augmented Reality," 2017 IEEE international conference on systems, man, and cybernetics (SMC). IEEE, 2017.
  20. P. D. A. S. a. A. K. M. Bhargava, "Speech enabled integrated AR-based multimodal language translation," 2016 Conference on Advances in Signal Processing (CASP), pp. 226-230, 2016.
  21. W. W.-T. L. a. H.-P. Y. Lin, "How the multimodal media in augmented reality affects museum learning Experience.," 2019 12th Asia Pacific Workshop on Mixed and Augmented Reality (APMAR). IEEE., 2019.
  22. W. Huang, L. Alem, F. Tecchia and H. B.-L. Duh, "Augmented 3D hands: a gesture-based mixed reality system for distributed collaboration.," Journal on Multimodal User Interfaces, pp. 77-89, 2018.
  23. A. Qeshta, Qeshta, A. "The Impact of Using Two Patterns of Augmented Reality in Improving the Scientific Concept and Scientific Sense in Science among Female Seventh Grade." , Gaza: The Islamic University of Gaza, 2018.
  24. N. Nguyen, Developing a multiplayer AR game using AR Foundation and Unity., 2020.
  25. R. R. Z. A. S. a. S. S. A. Asaad, "Proposed System for Education Augmented Reality Self English Learning.," Academic Journal of Nawroz University , pp. 27-32, 2019.
  26. H. a. N. G.-Y. Kose, "Augmented reality (AR) as a learning material in special needs education.," Education and Information Technologies , pp. 1921-1936, 2021.
  27. B. a. J. N. Lam, Creating Good User Experience in a Hand-Gesture-Based Augmented Reality Game., 2019.
  28. ManoMotion SDK Community Edition SDK CE, "SDK CE by ManoMotion," 2021. [Online]. Available: https://developer.manomotion.com/wpcontent/uploads/2021/05/sdk-ce-37.pdf.
  29. L. a. J. A. Crocker, "Introduction to classical and modern test theory.," ERIC, 2006.
  30. R. J. . Gregory, "Psychological testing: History, principles, and applications.," Allyn & Bacon, 2015a.
  31. P. . Good, Permutation tests: a practical guide to resampling methods for testing hypotheses., Springer Science & Business Media, 2013