• Title/Summary/Keyword: User experience evaluation methodology

Search Result 20, Processing Time 0.027 seconds

Development of a User Experience Evaluation Methodology for Smart Safety Living Lab (스마트안전 리빙랩 환경에서의 사용자경험 평가를 위한 방법론 개발)

  • Choi, Jae-Rim;Ryu, Do-Hyeon;Kim, Kwang-Jae;Yun, Jung-Min;Kim, Min-Sun
    • Journal of Korean Society for Quality Management
    • /
    • v.49 no.2
    • /
    • pp.127-143
    • /
    • 2021
  • Purpose: Smart Safety Living Lab is a Living Lab facility, constructed and operated by KITECH in Korea, to support the user experience(UX) evaluation, planning and certification of smart safety products and services. The purpose of this study is to develop a UX evaluation methodology that accommodates the characteristics of the Living Lab and smart safety products and services for a systematic and efficient UX evaluation in the Smart Safety Living Lab. Methods: A generic model of UX evaluation was first derived based on a review of related literature. Then, the generic model is revised to accommodate the characteristics of the Smart Safety Living Lab and smart safety products and services, resulting in the UX Evaluation Methodology for Smart Safety Living Lab (SSLL-UXEM). Results: The developed SSLL-UXEM consists of a structured process for UX evaluation, a guideline for conducting each step of the process, and a set of forms for recording the major evaluation results in each step. Conclusion: SSLL-UXEM can help to enhance the efficiency of the UX evaluation process and the consistency of the UX evaluation results. SSLL-UXEM is also expected to serve as a basis for UX evaluation in various living lab environments in the future.

User Experience Evaluation of Menstrual Cycle Measurement Application Using Text Mining Analysis Techniques (텍스트 마이닝 분석 기법을 활용한 월경주기측정 애플리케이션 사용자 경험 평가)

  • Wookyung Jeong;Donghee Shin
    • Journal of the Korean Society for information Management
    • /
    • v.40 no.4
    • /
    • pp.1-31
    • /
    • 2023
  • This study conducted user experience evaluation by introducing various text mining techniques along with topic modeling techniques for mobile menstrual cycle measurement applications that are closely related to women's health and analyzed the results by combining them with a honeycomb model. To evaluate the user experience revealed in the menstrual cycle measurement application review, 47,117 Korean reviews of the menstrual cycle measurement application were collected. Topic modeling analysis was conducted to confirm the overall discourse on the user experience revealed in the review, and text network analysis was conducted to confirm the specific experience of each topic. In addition, sentimental analysis was conducted to understand the emotional experience of users. Based on this, the development strategy of the menstrual cycle measurement application was presented in terms of accuracy, design, monitoring, data management, and user management. As a result of the study, it was confirmed that the accuracy and monitoring function of the menstrual cycle measurement of the application should be improved, and it was observed that various design attempts were required. In addition, the necessity of supplementing personal information and the user's biometric data management method was also confirmed. By exploring the user experience (UX) of the menstrual cycle measurement application in-depth, this study revealed various factors experienced by users and suggested practical improvements to provide a better experience. It is also significant in that it presents a methodology by combines topic modeling and text network analysis techniques so that researchers can closely grasp vast amounts of review data in the process of evaluating user experiences.

The Method of User Experience Evaluation for Virtual Living Space (가상현실 주거공간을 위한 사용자경험 평가 방법)

  • Kim, Mi-Sun;Paek, Hee-Won;Seo, Hye-Ran;Ko, Il-Ju
    • Asia-pacific Journal of Multimedia Services Convergent with Art, Humanities, and Sociology
    • /
    • v.7 no.12
    • /
    • pp.795-806
    • /
    • 2017
  • In this paper, we propose a user experience evaluation method for virtual reality living space contents and apply it to prototype simulations to measure user experience. The user evaluation method proposed in this paper is based on the existing user evaluation methodology, but it is established by changing the experience and evaluation factors to match the characteristics of the virtual reality living space. The generated virtual reality living space simulation is configured similar to real space and is placed real objects that the user actually uses in each space, so that users can feel reality. The subjects performed the tasks according to the given scenarios and then evaluated the simulation. As a result of the evaluation, the main goal of the simulation, the items that make the users feel realistic, were lower than our expectations. This means that future simulations need to be modified in a way that adds or expands elements that enhance the realism. In this way, the UX evaluation method proposed in this paper is expected to provide a more objective indicator to the content developer or designer in order to grasp the user's demand in the virtual reality living space.

A Study on Improving Experience of Visiting Obstetrics and Gynecology of Single Women - Using Service Design Methodology (미혼 여성의 산부인과 방문 경험 개선 연구 - 서비스 디자인 방법론을 활용하여)

  • Kim, Ye Bin;Chon, Woo Jeong
    • Journal of Korea Multimedia Society
    • /
    • v.24 no.12
    • /
    • pp.1693-1707
    • /
    • 2021
  • The purpose of this study was to improve the experience of visiting obstetrics and gynecology of single women. After analyzing previous studies on Korean single women's perception of visiting obstetrics and gynecology, Contextual Interviews and Cultural Probes were conducted on single women in their 20s who visited obstetrics and gynecology. Based on this, personas were constructed to solidify the direction of problem solving by identifying the behavioral patterns and characteristics of single women. In this study, factors that hinder unmarried women's visits to obstetrics and gynecology and improvement measures were derived based on the information obtained using service design tools such as User Journey Mapping and Stakeholders' Map. Afterwards, a preference survey was conducted to increase the persuasiveness of the proposed method. The follow-up research task is to produce and propose the derived solution as a prototype that can be used in the actual field, and then proceed with user evaluation.

Using the METHONTOLOGY Approach to a Graduation Screen Ontology Development: An Experiential Investigation of the METHONTOLOGY Framework

  • Park, Jin-Soo;Sung, Ki-Moon;Moon, Se-Won
    • Asia pacific journal of information systems
    • /
    • v.20 no.2
    • /
    • pp.125-155
    • /
    • 2010
  • Ontologies have been adopted in various business and scientific communities as a key component of the Semantic Web. Despite the increasing importance of ontologies, ontology developers still perceive construction tasks as a challenge. A clearly defined and well-structured methodology can reduce the time required to develop an ontology and increase the probability of success of a project. However, no reliable knowledge-engineering methodology for ontology development currently exists; every methodology has been tailored toward the development of a particular ontology. In this study, we developed a Graduation Screen Ontology (GSO). The graduation screen domain was chosen for the several reasons. First, the graduation screen process is a complicated task requiring a complex reasoning process. Second, GSO may be reused for other universities because the graduation screen process is similar for most universities. Finally, GSO can be built within a given period because the size of the selected domain is reasonable. No standard ontology development methodology exists; thus, one of the existing ontology development methodologies had to be chosen. The most important considerations for selecting the ontology development methodology of GSO included whether it can be applied to a new domain; whether it covers a broader set of development tasks; and whether it gives sufficient explanation of each development task. We evaluated various ontology development methodologies based on the evaluation framework proposed by G$\acute{o}$mez-P$\acute{e}$rez et al. We concluded that METHONTOLOGY was the most applicable to the building of GSO for this study. METHONTOLOGY was derived from the experience of developing Chemical Ontology at the Polytechnic University of Madrid by Fern$\acute{a}$ndez-L$\acute{o}$pez et al. and is regarded as the most mature ontology development methodology. METHONTOLOGY describes a very detailed approach for building an ontology under a centralized development environment at the conceptual level. This methodology consists of three broad processes, with each process containing specific sub-processes: management (scheduling, control, and quality assurance); development (specification, conceptualization, formalization, implementation, and maintenance); and support process (knowledge acquisition, evaluation, documentation, configuration management, and integration). An ontology development language and ontology development tool for GSO construction also had to be selected. We adopted OWL-DL as the ontology development language. OWL was selected because of its computational quality of consistency in checking and classification, which is crucial in developing coherent and useful ontological models for very complex domains. In addition, Protege-OWL was chosen for an ontology development tool because it is supported by METHONTOLOGY and is widely used because of its platform-independent characteristics. Based on the GSO development experience of the researchers, some issues relating to the METHONTOLOGY, OWL-DL, and Prot$\acute{e}$g$\acute{e}$-OWL were identified. We focused on presenting drawbacks of METHONTOLOGY and discussing how each weakness could be addressed. First, METHONTOLOGY insists that domain experts who do not have ontology construction experience can easily build ontologies. However, it is still difficult for these domain experts to develop a sophisticated ontology, especially if they have insufficient background knowledge related to the ontology. Second, METHONTOLOGY does not include a development stage called the "feasibility study." This pre-development stage helps developers ensure not only that a planned ontology is necessary and sufficiently valuable to begin an ontology building project, but also to determine whether the project will be successful. Third, METHONTOLOGY excludes an explanation on the use and integration of existing ontologies. If an additional stage for considering reuse is introduced, developers might share benefits of reuse. Fourth, METHONTOLOGY fails to address the importance of collaboration. This methodology needs to explain the allocation of specific tasks to different developer groups, and how to combine these tasks once specific given jobs are completed. Fifth, METHONTOLOGY fails to suggest the methods and techniques applied in the conceptualization stage sufficiently. Introducing methods of concept extraction from multiple informal sources or methods of identifying relations may enhance the quality of ontologies. Sixth, METHONTOLOGY does not provide an evaluation process to confirm whether WebODE perfectly transforms a conceptual ontology into a formal ontology. It also does not guarantee whether the outcomes of the conceptualization stage are completely reflected in the implementation stage. Seventh, METHONTOLOGY needs to add criteria for user evaluation of the actual use of the constructed ontology under user environments. Eighth, although METHONTOLOGY allows continual knowledge acquisition while working on the ontology development process, consistent updates can be difficult for developers. Ninth, METHONTOLOGY demands that developers complete various documents during the conceptualization stage; thus, it can be considered a heavy methodology. Adopting an agile methodology will result in reinforcing active communication among developers and reducing the burden of documentation completion. Finally, this study concludes with contributions and practical implications. No previous research has addressed issues related to METHONTOLOGY from empirical experiences; this study is an initial attempt. In addition, several lessons learned from the development experience are discussed. This study also affords some insights for ontology methodology researchers who want to design a more advanced ontology development methodology.

The Reminding Words Evaluation Method for Icon Design (연상단어 평가를 통한 아이콘 디자인 연구)

  • Ha, Kwang Soo
    • The Journal of the Korea Contents Association
    • /
    • v.14 no.10
    • /
    • pp.64-72
    • /
    • 2014
  • As the importance of interface became emphasized recently, function of icon as an element that helps operation and formative function as an aesthetic element are becoming emphasized. Specially, various research methodologies for measuring the understanding degree of users are being attempted in functional aspect, however, they are facing limitations as user evaluation method of quantitative investigation for statistical analysis became overly dependent on the consent from majority. To overcome such limitation, a methodology of measuring sensibility as aesthetic impression or individuality based on the system of emotion or system of understanding of users and designing icon based on the measurement is being attempted continuously. Therefore, this study suggested an evaluation method using reminding words and conducted an empirical study to apply results on the icon design by applying the evaluation method on icon design of printing products to measure the user sensibility and understanding level. Through this process, this study intended to research summaries on the design evaluation method which based on the evaluation of user sensibility and understanding level and to debate effective design methodologies through the process of applying these summaries on the project. The intention was to progress discussion from practical application perspectives apart from the textbook type discussion by applying a research methodology through concrete cases. This study objectified sensibility and understanding level of users in the icon design aspect, suggested a methodology that can be applied on the design, and it is expected to be used when progressing the icon and GUI design.

Research on Application of Service Design Methodology in IOT Environment

  • Kim, Byung-Taek;Cho, Yun-Sung
    • Journal of the Korea Society of Computer and Information
    • /
    • v.22 no.3
    • /
    • pp.53-60
    • /
    • 2017
  • The purpose of this study is to provide empirical and quantitative analysis on user's perceived privacy, security, and user satisfaction when providing visualization information about objects and service provider behaviors that users can not perceive in internet service process. Through previous research, we have examined the importance of privacy and security factors as a key factor to be considered for the characteristics of the Internet of things and the Internet of things. In addition, service blueprint, which is one of the service design methodologies to examine the flow of service usage in providing Internet service of things, was examined. In the flow of things internet service utilization, it is found that the things that are out of the user's cognitive area and the behavior of the service provider take up a large part. Therefore, the hypothesis that the trust of the Internet service security and the satisfaction of the user experience can be improved by providing the security visualization information about the behavior of the object and the invisible service provider in the non-contact aspect of the user and the object. In order to verify the hypothesis, we conducted experiments and questionnaires on the use of virtual objects' internet environment and conducted statistical analysis based on them. As a result, it was analyzed that visual information feedback on non - contact and invisible objects and service provider's behaviors had a positive effect on user's perceived privacy, security, and satisfaction. In addition, we conclude that it can be used as a service design evaluation tool to eliminate psychological anxiety about security and to improve satisfaction in internet service design. We hope that this research will be a great help for the research on application method of service design method in Internet environment of objects.

A Study on Quality evaluation Methodology Establishment of Anti-Virus Software based on the Real Test Environment (리얼 테스트 환경 기반의 안티바이러스 소프트웨어의 품질평가 방법론 정립에 관한 연구)

  • Maeng, Doo-Iyel;Park, Jong-Kae;Kim, Sung-Joo
    • The Journal of Korean Institute of Communications and Information Sciences
    • /
    • v.35 no.3B
    • /
    • pp.440-452
    • /
    • 2010
  • For an evaluation of the software product, the national/international organizations and labs have been studying various methodologies for the quality on the basis of ISO/IEC Quality Assurance System, but they still have many issues in evaluation of the anti-virus software that has special characteristics of complexity. In this paper, to establish a methodology of the quality evaluation for the anti-virus software, fulfilling the requirements more than reasonable level, a process to draw the evaluation items and quantification was established. And the information of weight was objectified by analyzing the relative magnitude between each factors. Based on the defined information (evaluation item, weight), conducting the quality evaluations for 70 kinds of open anti-virus software collected from the portal sites in the real test environment, and as a result of the positive analysis with user's long-term experience, this paper justifies the evaluation item and the weight.

A Study on the Method of Evaluation concerning Mobile Interface Design on the Basis of the Online Workbook (온라인 워크북을 활용한 모바일 인터페이스 디자인 평가 방법에 관한 연구)

  • Choi, Han-Jin;Lee, Seung-Hun;Ji, Ju-Min;Kim, Ae-Young;Kim, Jung-Ha
    • 한국HCI학회:학술대회논문집
    • /
    • 2006.02b
    • /
    • pp.191-196
    • /
    • 2006
  • 새로운 휴대폰 기기가 하루가 다르게 개발되어 상품화 되는 가운데 사용자의 시각에 입각한 사용성 평가는 앞으로 새롭게 선보일 휴대폰 기기의 방향성을 제시해 줄 수 있는 중요한 척도이다. 특히 근 몇 년간 다양한 기능이 convergence 된 휴대폰들이 늘어남에 따라 사용자 관점에서의 평가 방법에 있어, 실제 사용자는 예측할 수 없고 셀 수 없는 다양한 상황과 환경 속에서 휴대폰 기기를 사용하게 된다는 사실을 주목해야 할 것이다. 따라서 본 연구에서는 끊임없이 변화하는 휴대폰 기기로 인하여 실제 user 가 자신의 특성과 습관, 시시각각 다르게 처하는 환경과 함께 야기되는 휴대폰에 대한 use experience 를 수집하고 분석하는 적합한 방법론을 새로이 제시하고, 이를 사례 연구를 통하여 검증함으로써 사용자 경험을 수집하는데 유용한 새로운 방법론을 개발하는데 그 목적이 있다. 이러한 사용성 테스트의 새로운 분석 방법으로, 특정 휴대폰의 웹 리뷰사이트, 인터넷 동우회와 같은 온라인 커뮤니티상에서 테스트에 응할 여지가 있는 expert user 를 엄선하여 선정 하였으며, 그들에게 휴대폰을 사용하면서 겪게 되는 경험이나 문제점을 휴대폰의 어떠한 부분에서 언제, 어떠한 식으로 발생되었는지에 대하여 work book 의 형식을 통하여 기록하고 다시 그것을 바탕으로 In-depth interview 함으로써 사용자의 경험의 축적에 따라 사용성을 평가하는 프로세스를 새로이 개발하고 적용하였다.

  • PDF

Delphi Research on Usability Test Framework of Metaverse Platform - Case of Roblox, Zepeto, and Gathertown (메타버스 플랫폼 사용성 평가체계 구축에 관한 델파이연구 - 로블록스, 제페토, 게더타운 사례를 중심으로)

  • Lee, Han Jin;Gu, Hyun Hee
    • The Journal of the Korea Contents Association
    • /
    • v.22 no.9
    • /
    • pp.179-193
    • /
    • 2022
  • Amid the explosive growth of various metaverse platforms, there is no unified indicator to measure, analyze, and evaluate based on customer experience. Therefore, the usability evaluation factors in metaverse were identified through a heuristic methodology and literature review, to evaluate the metaverse, a two-to three-dimensional virtual world platform. A measurable system was established by subdividing 20 items in 5 fields, including user control, information structure, design and content, and usage environment, derived through Delphi technique. Based on this, after experiencing the actual contents of major metaverse platforms such as Roblox and Zepeto, usability was evaluated and comparative verification was conducted. As a result, it was estimated that metaverse user experience could be improved as its utility was derived relatively high in terms of user control and content. This study constitutes a theoretical contribution by extending the usability evaluation system, which has been widely used in the field of service design, to the fields of extended reality and mixed reality. At the same time, it has practical key findings of providing basic judgment standards to stakeholders in the metaverse field, as well as policy implications for digital capability enhancement and industry revitalization.