• Title/Summary/Keyword: User Feedback Evaluation

Search Result 78, Processing Time 0.024 seconds

Understanding of Generative Artificial Intelligence Based on Textual Data and Discussion for Its Application in Science Education (텍스트 기반 생성형 인공지능의 이해와 과학교육에서의 활용에 대한 논의)

  • Hunkoog Jho
    • Journal of The Korean Association For Science Education
    • /
    • v.43 no.3
    • /
    • pp.307-319
    • /
    • 2023
  • This study aims to explain the key concepts and principles of text-based generative artificial intelligence (AI) that has been receiving increasing interest and utilization, focusing on its application in science education. It also highlights the potential and limitations of utilizing generative AI in science education, providing insights for its implementation and research aspects. Recent advancements in generative AI, predominantly based on transformer models consisting of encoders and decoders, have shown remarkable progress through optimization of reinforcement learning and reward models using human feedback, as well as understanding context. Particularly, it can perform various functions such as writing, summarizing, keyword extraction, evaluation, and feedback based on the ability to understand various user questions and intents. It also offers practical utility in diagnosing learners and structuring educational content based on provided examples by educators. However, it is necessary to examine the concerns regarding the limitations of generative AI, including the potential for conveying inaccurate facts or knowledge, bias resulting from overconfidence, and uncertainties regarding its impact on user attitudes or emotions. Moreover, the responses provided by generative AI are probabilistic based on response data from many individuals, which raises concerns about limiting insightful and innovative thinking that may offer different perspectives or ideas. In light of these considerations, this study provides practical suggestions for the positive utilization of AI in science education.

Analysis of Hazard Factors for Domestic General Purpose Ventilator using Usability Assessment (사용적합성 평가를 적용한 국산 범용인공호흡기의 위험요인 분석)

  • Gyeongmin Kwon;Seung hee Kim;You Rim Kim;Won Seuk Jang
    • Journal of Biomedical Engineering Research
    • /
    • v.45 no.1
    • /
    • pp.10-19
    • /
    • 2024
  • The purpose of this study is to conduct a summative evaluation of the usability of a general-purpose ventilator to determine whether it can be used for its intended purpose in the intended environment by the intended user and to find possible errors in use. The importance of ventilators has increased due to the accelerated aging of the population and the impact of the pandemic. In addition, patients who require ventilators are often in critical condition, so even a small error in use can be fatal. Therefore, it is important to ensure that the ventilator has sufficient stability and can be used satisfactorily without inconvenience to the user. In this study, we conducted a usability test with 17 respiratory nurses with more than 3 years of experience using the ventilator. We analyzed the task success rate, satisfaction, and opinions of the intended users while going through a total of 17 scenarios. Satisfaction was captured through an ASQ questionnaire and subjective opinions were captured through a detailed opinion questionnaire. The results showed a high level of satisfaction with an average score of 6.3 for the use scenarios. Evaluators expressed satisfaction with the overall visibility and versatility of the features, but noted that improvements were needed for calibration tasks with low task success rates. As the calibration method is different from other equipment, it was suggested that specific explanations of the calibration method and the picture that appears when calibrating are needed, and that if relevant training is provided, the equipment can be used without problems. If the usability evaluation is not limited to securing efficiency and satisfaction from the intended users, but also continuously receives feedback from users to prepare for use in emergency environments such as pandemic situations, it will be very helpful to seize opportunities such as emergency authorization in future situations, and ultimately contribute to patient safety by reducing use errors.

Usability Testing of Digital Pressure Bio-feedback for Spinal Rehabilitation Exercise (척추재활운동을 위한 디지털 압력바이오피드백 장치의 사용성 평가)

  • Kim, Tea-Ho;Oh, Do-Bong;Kim, Da-Yeon
    • Journal of rehabilitation welfare engineering & assistive technology
    • /
    • v.11 no.3
    • /
    • pp.119-126
    • /
    • 2017
  • In the clinical setting, the pressure bio-feedback device is used for the spinal rehabilitation of patients with back pain, but it has several disadvantages. The purpose of this study was to develop a digitalized pressure biofeedback system that provides precise exercise method and posture in real time during the spinal rehabilitation exercise by sensing and monitoring body movements and balance of users and providing biofeedback to users. After that, the usability testing for a digitalized pressure biofeedback system will be conducted to identify problems such as safety, performance, operability, and satisfaction, and suggest improvement directions. A total of 33 subjects were participated in the usability testing. The experts group and the users group evaluated the developed digitalized pressure biofeedback system on a scale of 5 points after using the equipment. In the user group, safety was 3.59, operability was 4.38, satisfaction was 4.49. In the expert group, safety was 2.86, operability was 3.91, and performance was 4.28. Based on the usability evaluation, if the problems of stability of the cradle for tablet PC, air injection, screen display, etc. are solved, it becomes a exercise device capable of accurately exercising and evaluating the function of the spine by checking its own motion state while the spinal stabilization exercise.

An Investigation of Information Usefulness of Google Scholar in Comparison with Web of Science (Google Scholar의 학술정보 검색을 위한 정보 유용성 비교연구)

  • Kim, Hyunjung
    • Journal of the Korean BIBLIA Society for library and Information Science
    • /
    • v.25 no.3
    • /
    • pp.215-234
    • /
    • 2014
  • The purpose of this study is to investigate whether Google Scholar (GS) can substitute Web of Science (WoS) for those who don't have access to the subscription-based indexing service and if users feel GS is useful for scholarly information. To achieve the research purpose, the study evaluates both quantitative and qualitative aspects of the two databases. The major results through statistical analysis show that GS indexes much more records and citations for LIS journals than WoS(p < .01), but users' feedback about GS is not better than those about WoS.

A Study on Users' Perception of Reference Services in National Archives of Korea (국가기록원 기록정보서비스에 대한 이용자 인식에 관한 연구)

  • Kim, Jihyun
    • Journal of Korean Society of Archives and Records Management
    • /
    • v.12 no.1
    • /
    • pp.167-187
    • /
    • 2012
  • This study investigated the perception and experience of users who visited in Seoul office of the National Archives, a branch of the National Archives of Korea(NAK). For collecting data, the study utilized the Researcher Survey Toolkit developed by Archival Metrics Project and revised the questionnaire to reflect services of NAK. Questionnaires completed by 47 users were collected and analyzed, and interviews with one survey respondent and two staff were performed. It was found that the purpose of the visit is mostly to identify records that prove land ownership of users' ancestors. Types of records frequently used were also those concerning lands and forests. User feedback on the staff was generally positive, and users perceived them to be helpful and kind. However, there was an opinion that the staff did not provide accurate information about whether NAK hold records that the user wanted to find. The staff also said that it was challenging to confirm where requested records are located when providing services for users. User evaluation on the usefulness and the ease of use of NAK portal services was not very positive, and it was perceived to be very inconvenient to search in the portal. Overall, users were satisfied with the services of NAK, although some users suggested that services need be provided efficiently, and NAK must clarify the criteria for the non-disclosure of public records.

Summative Usability Assessment of Software for Ventilator Central Monitoring System (인공호흡기 중앙감시시스템 소프트웨어의 사용적합성 총괄평가)

  • Ji-Yong Chung;You Rim Kim;Wonseuk Jang
    • Journal of Biomedical Engineering Research
    • /
    • v.44 no.6
    • /
    • pp.363-376
    • /
    • 2023
  • According to the COVID-19, development of various medical software based on IoT(Internet of Things) was accelerated. Especially, interest in a central software system that can remotely monitor and control ventilators is increasing to solve problems related to the continuous increase in severe COVID-19 patients. Since medical device software is closely related to human life, this study aims to develop central monitoring system that can remotely monitor and control multiple ventilators in compliance with medical device software development standards and to verify performance of system. In addition, to ensure the safety and reliability of this central monitoring system, this study also specifies risk management requirements that can identify hazardous situations and evaluate potential hazards and confirms the implementation of cybersecurity to protect against potential cyber threats, which can have serious consequences for patient safety. As a result, we obtained medical device software manufacturing certificates from MFDS(Ministry of Food and Drug Safety) through technical documents about performance verification, risk management and cybersecurity application.The purpose of this study is to conduct a usability assessment to ensure that ergonomic design has been applied so that the ventilator central monitoring system can improve user satisfaction, efficiency, and safety. The rapid spread of COVID-19, which began in 2019, caused significant damage global medical system. In this situation, the need for a system to monitor multiple patients with ventilators was highlighted as a solution for various problems. Since medical device software is closely related to human life, ensuring their safety and satisfaction is important before their actual deployment in the field. In this study, a total of 21 participants consisting of respiratory staffs conducted usability test according to the use scenarios in the simulated use environment. Nine use scenarios were conducted to derive an average task success rate and opinions on user interface were collected through five-point Likert scale satisfaction evaluation and questionnaire. Participants conducted a total of nine use scenario tasks with an average success rate of 93% and five-point Likert scale satisfaction survey showed a high satisfaction result of 4.7 points on average. Users evaluated that the device would be useful for effectively managing multiple patients with ventilators. However, improvements are required for interfaces associated with task that do not exceed the threshold for task success rate. In addition, even medical devices with sufficient safety and efficiency cannot guarantee absolute safety, so it is suggested to continuously evaluate user feedback even after introducing them to the actual site.

Evaluation Criteria for Appropriateness of Bicycle Riding Path Considering Cyclist's Trip Purposes (자전거 이용자의 통행목적을 고려한 주행경로 적정성 평가지표 개발)

  • Kim, Eui-Jin;Kim, Dong-Kyu
    • The Journal of The Korea Institute of Intelligent Transport Systems
    • /
    • v.15 no.4
    • /
    • pp.12-25
    • /
    • 2016
  • The purpose of this study is providing an evaluation criteria for appropriateness of bicycle riding path considering trip purpose based on cyclist's response of importance and satisfaction. The survey presents a quantified criteria of evaluating appropriateness of the bicycle path by investigating difference of impotance and satisfaction between two purposes among 5 selected influence factors about bicycle lane and bicycle parking facility, and in case of commuting purpose, information of destination is additionally considered. All the influence factors are analyzed by Analytical Hierarchy Process (AHP) which yields importance. For the factors which have a huge difference between two purposes, evaluation criteria using a GIS Data of respondent's path and satisfaction of each factors is developed, and other factors are made it by reviewing literature. The importance analyzed by AHP and evaluation criteria can provide a path based LOS for cyclist, and this information can be improved through user's response from app or search engine in the future. and by considering individual's evaluation, it can provide individually specified information.

An Web-based Training of a short bamboo flute performance by using UCC (UCC를 활용한 단소 실기 원격 교육)

  • Lee, Yong-Bae;Lim, Sung-Joon
    • Journal of The Korean Association of Information Education
    • /
    • v.11 no.4
    • /
    • pp.471-482
    • /
    • 2007
  • These days UCC(User-created content) is being made and shared increasingly in entertainment and sports area, but its life cycle seems to be very short and the cases that it is used for an education or a learning purposes are not common yet. In this study a new methodology is suggested for adapting a UCC to a distance education. A teacher upload the movie that he or she made for the distance education system, so the students can carry out the self-centered learning procedure. After that, the students send their own movie files to the teacher, and get a feedback from the teacher as a evaluation of the course. In this study a distance education system was established as a prototype, and a short bamboo flute class was chosen for this study from the specialty developmental education program of the elementary school. According to the result of the questionnaire the students thought that their performance skill was improved a lot and they were satisfied with the learning program and the method of evaluation. They also answered that their skills dealing with a camera, a camcorder and a computer got much better. Moreover, most of the students thought that the relationships with their friends and their parents got better also because they spent lots of time together making and watching the movie files for this education program.

  • PDF

Application of Paper-prototyping Method for Touch Interface Design Evaluation (터치 입력방식 디자인 평가를 위한 페이퍼 프로토타이핑 기법 적용)

  • Shin, Kyung-Jin;Lee, Tae-Il
    • 한국HCI학회:학술대회논문집
    • /
    • 2009.02a
    • /
    • pp.888-894
    • /
    • 2009
  • Recently, touch screen systems with direct touch interfaces are prevailing in digital products such as MP3's, DMB's, PDA's and navigators, which give rise to new interface issues related to speed, feedback and accuracy, and consider additional considerations for solving those issues in evaluation processes. Although paper prototyping method, the one of applicable methods, is efficient ways to evaluate the touch interface, it still requires a certain amount of adaptations for touch interfaces. This study aims to identify emerged issues regarding to paper-prototyping method for the evaluation of touch interface, to come up with alternatives for applying the method. The study started with analyzing the touch interface in digital products, and suggests the 'Touch Wheel' interface based on mainly used interface. 'Touch Wheel' interface allows the user searching for her/his favorite songs by rolling 'wheel-like' menu list. The Userbility test was conducted by 24 MP3 users with 4 times, and based on such conditions as materials, making methods and evaluating process of paper prototyping for evaluating the touch interface, effective alternatives were considered in each evaluation steps. As a result, paper prototypes for evaluating the touch interface should give active operations to subjects by offering direct experience, so as to provide subjects with proper feedbacks and for designers to collect effective alternatives for expected problems in initial interface design process.

  • PDF

Query Expansion based on Word Sense Community (유사 단어 커뮤니티 기반의 질의 확장)

  • Kwak, Chang-Uk;Yoon, Hee-Geun;Park, Seong-Bae
    • Journal of KIISE
    • /
    • v.41 no.12
    • /
    • pp.1058-1065
    • /
    • 2014
  • In order to assist user's who are in the process of executing a search, a query expansion method suggests keywords that are related to an input query. Recently, several studies have suggested keywords that are identified by finding domains using a clustering method over the documents that are retrieved. However, the clustering method is not relevant when presenting various domains because the number of clusters should be fixed. This paper proposes a method that suggests keywords by finding various domains related to the input queries by using a community detection algorithm. The proposed method extracts words from the top-30 documents of those that are retrieved and builds communities according to the word graph. Then, keywords representing each community are derived, and the represented keywords are used for the query expansion method. In order to evaluate the proposed method, we compared our results to those of two baseline searches performed by the Google search engine and keyword recommendation using TF-IDF in the search results. The results of the evaluation indicate that the proposed method outperforms the baseline with respect to diversity.