Browse > Article
http://dx.doi.org/10.7469/JKSQM.2021.49.2.127

Development of a User Experience Evaluation Methodology for Smart Safety Living Lab  

Choi, Jae-Rim (Department of Industrial and Management Engineering, Pohang University of Science and Technology)
Ryu, Do-Hyeon (Department of Industrial and Management Engineering, Pohang University of Science and Technology)
Kim, Kwang-Jae (Department of Industrial and Management Engineering, Pohang University of Science and Technology)
Yun, Jung-Min (Korea National Industrial Convergence Center, Korea Institute of Industrial Technology)
Kim, Min-Sun (Korea National Industrial Convergence Center, Korea Institute of Industrial Technology)
Publication Information
Journal of Korean Society for Quality Management / v.49, no.2, 2021 , pp. 127-143 More about this Journal
Abstract
Purpose: Smart Safety Living Lab is a Living Lab facility, constructed and operated by KITECH in Korea, to support the user experience(UX) evaluation, planning and certification of smart safety products and services. The purpose of this study is to develop a UX evaluation methodology that accommodates the characteristics of the Living Lab and smart safety products and services for a systematic and efficient UX evaluation in the Smart Safety Living Lab. Methods: A generic model of UX evaluation was first derived based on a review of related literature. Then, the generic model is revised to accommodate the characteristics of the Smart Safety Living Lab and smart safety products and services, resulting in the UX Evaluation Methodology for Smart Safety Living Lab (SSLL-UXEM). Results: The developed SSLL-UXEM consists of a structured process for UX evaluation, a guideline for conducting each step of the process, and a set of forms for recording the major evaluation results in each step. Conclusion: SSLL-UXEM can help to enhance the efficiency of the UX evaluation process and the consistency of the UX evaluation results. SSLL-UXEM is also expected to serve as a basis for UX evaluation in various living lab environments in the future.
Keywords
User experience evaluation methodology; Living Lab; Smart safety products and services; Smart Safety Living Lab;
Citations & Related Records
연도 인용수 순위
  • Reference
1 Jaspers, M. W. 2009. A Comparison of Usability Methods for Testing Interactive Health Technologies: Methodological Aspects and Empirical Evidence. International Journal of Medical Informatics 78(5):340-353.   DOI
2 Winckler, M., Bernhaupt, R., and Bach, C. 2016. Identification of UX Dimensions for Incident Reporting Systems with Mobile Applications in Urban Contexts: a Longitudinal Study. Cognition, Technology & Work 18(4): 673-694.   DOI
3 Kim, H., Han, S., Park, J., and Park, W. 2015. How User Experience Changes Over Time: A Case Study of Social Nnetwork Services. Human Factors and Ergonomics in Manufacturing & Service Industries 25(6):659-673.   DOI
4 Laugwitz, B., Held, T., and Schrepp, M. 2008. Construction and Evaluation of a User Experience Questionnaire. In Symposium of the Austrian HCI and Usability Engineering Group 63-76.
5 Nawaz, A., Skjaeret, N., Ystmark, K., Helbostad, J. L., Vereijken, B., and Svanaes, D. 2014, Assessing Seniors' User Experience (UX) of Exergames for Balance Training. In Proceedings of the 8th Nordic Conference on Human-computer Interaction: Fun, Fast, Foundational 578-587.
6 Parker, R., Vitalis, A., Walker, R., Riley, D., and Pearce, H. G. 2017. Measuring Wildland Fire Fighter Performance with Wearable Technology. Applied Ergonomics 59:34-44.   DOI
7 Kaasinen, E., Roto, V., Hakulinen, J., Heimonen, T., Jokinen, J. P., Karvonen, H., ... and Turunen, M. 2015. Defining User Experience Goals to Guide the Design of Industrial Systems. Behaviour & Information Technology 34(10):976-991.   DOI
8 Nesterova, N., and Quak, H. 2015. CITYLAB Deliverable 3.1. Practical Guidelines for Establishing and Running a Ccity Logistics Living Laboratory. Available from: https://civitas.eu/tool-inventory/city-logistics- living-lab-handbook.
9 Rajeshkumar, S., Omar, R., and Mahmud, M. 2013. Taxonomies of User Experience (UX) Evaluation Methods. In 2013 International Conference on Research and Innovation in Information Systems 533-538.
10 Leminen, S., Westerlund, M., and Nystrom, A. G. 2012. Living Labs as Open-Innovation Networks. Technology Innovation Management Review 2(9):6-11.   DOI
11 Seong, J., Song, W., Jung, B., Choi, C., Yoon, C., Jeong, S., and Han, K. 2017. Current Status of Korean Living Labs and Its Development Plan. Sci Technol Policy Institute 9:1-194.
12 Stahlbrost, A., and Holst, M. 2012. The Living Lab Methodology Handbook. Social Informatics at Lulea University of Technology and CDT-Centre for Distance-spanning Technology
13 Vaziri, D. D., Aal, K., Ogonowski, C., Von Rekowski, T., Kroll, M., Marston, H. R., ... and Wulf, V. 2016. Exploring User Experience and Technology Acceptance for a Fall Prevention System: Results from a Randomized Clinical Trial and a Living Lab. European Review of Aging and Physical Activity 13(1): 6.   DOI
14 Desmet, P., Overbeeke, K., and Tax, S. 2001. Designing Products with Added Emotional Value: Development and Appllcation of an Approach for Research through Design. The Design Journal 4(1):32-47.   DOI
15 An, K. H., Lee, S. B., Lee, S. B., and Suh, Y. H. 2018. An Effect of O2O Service Users' Motivation on Loyalty through Expectation-Confirmation and Satisfaction. Journal of the Korean Society for Quality Management 46(4):923-938.   DOI
16 Bae, J. H., Park, J. H., Lee, H. N., and Choi, J. I. 2019. A Study of Factors Affecting Intention to Use of Using Technology-based Self-services for Smart Airport. Journal of Korean Society for Quality Management 47(4):795-806.
17 Bergvall-Kareborn, B., and Stahlbrost, A. 2009. Living Lab: An Open and Citizen-centric Approach for Iinnovation. International Journal of Innovation and Regional Development 1(4):356-370.   DOI
18 Chae, H. S., Ko, M. S., Kim, H. C., Kim, K. S., Choi, D. P., Kim, K. R., and Lee, K. S. 2017. Development Trends of Smart Personal Protective Equipment for Agricultural Health and Safety. Journal of the Ergonomics Society of Korea 36(6):677-691.   DOI
19 Cooper, R. G. 2008. Perspective: The Stage-gate® idea-to-launch process-Update, What's New, and Nexgen Systems. Journal of Product Innovation Mmanagement 25(3):213-232.   DOI
20 Dong, Y., and Liu, W. 2018. Research on UX Ealuation Method of Design Concept under Multi-modal Experience Scenario in the Earlier Design Stages. International Journal on Interactive Design and Manufacturing 12(2):505-515.   DOI
21 European Network of Living Labs. [Internet]. 2021 Feb 03. Available from: https://enoll.org/about-us/what-are-living-labs/.
22 Amidon, T. R., Williams, E. A., Lipsey, T., Callahan, R., Nuckols, G., and Rice, S. 2018. Sensors and Gizmos and Data, Oh My: Informating Ffirefighters' Personal Protective Equipment. Communication Design Quarterly Review 5(4):15-30.   DOI
23 Park, J., Han, S. H., Park, J., Park, J., Kwahk, J., Lee, M., and Jeong, D. Y. 2018. Development of a Web-based User Experience Evaluation Ssystem for Hhome Aappliances. International Journal of Industrial Ergonomics 67:216-228.   DOI
24 Turunen, M., Hakulinen, J., Melto, A., Heimonen, T., Laivo, T., and Hella, J. 2009. SUXES-user Experience Evaluation Method for Spoken and Multimodal Interaction. In Tenth Annual Conference of the International Speech Communication Association.
25 Hartson, R., and Pyla, P. S. 2012. The UX Book: Process and Guidelines for Ensuring a Quality User Experience. Elsevier.
26 Chen, H. E., Lin, Y. Y., Chen, C. H., and Wang, I. F. 2015. BlindNavi: A Navigation App for the Visually Iimpaired Smartphone User. In Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Hhuman Factors in Computing Systems 19-24.
27 Hassenzahl, M., and Tractinsky, N. 2006. User Experience-A Research Agenda. Behaviour & Iinformation Technology 25(2):91-97.   DOI
28 Jakobi, T., Ogonowski, C., Castelli, N., Stevens, G., and Wulf, V. 2017. The Catch (es) with Smart Home: Experiences of a Living Lab Field Study. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems 1620-1633.
29 Kim, M., Kim, P., and Yoon, J. 2020. Establishment and Application of Living Lab, Industrial Innovation Platform. IE Magazine 27(2):38-45.
30 Nascimento, I., Silva, W., Gadelha, B., and Conte, T. 2016. Userbility: A Technique for the Evaluation of User Experience and Usability on Mobile Aapplications. In International Conference on Human-Computer Interaction 372-383.
31 Schrepp, M. 2015. User Experience Questionnaire Handbook. All You Need to Know to Apply the UEQ Successfully in Your Project.
32 Wu, Y. H., Wrobel, J., Cornuet, M., Kerherve, H., Damnee, S., and Rigaud, A. S. 2014. Acceptance of an Assistive Robot in Older Adults: a Mixed-method Sstudy of Hhuman-robot Iinteraction Over a 1-month Priod in the Living Lab Setting. Clinical Interventions in Aging 9:801.   DOI
33 Jung, K. B., and Choi, S. B. 2020. The Effect of Employee Authenticity on Customer Loyalty via Rapport: A Moderated Mediation Model. Journal of the Korean Society for Quality Management 48(3):361-379.   DOI
34 Jang, R., Molesworth, B. R., Burgess, M., and Estival, D. 2014. Improving Communication in General Aviation through the Use of Noise Cancelling Headphones. Safety Science 62:499-504.   DOI
35 International Organization for Standardization. 2019. ISO 9241-210: 2019 (en) Ergonomics of Human-system Interaction-Part 210: Human-centred Design for Interactive Systems. Available from: https://www.iso.org/standard/77520.html