Journal of Biomedical Engineering Research (대한의용생체공학회:의공학회지)

The Korean Society of Medical and Biological Engineering (대한의용생체공학회)
권호 표지
DOI QR코드

DOI QR Code

Summative Usability Assessment of Software for Ventilator Central Monitoring System

인공호흡기 중앙감시시스템 소프트웨어의 사용적합성 총괄평가

  • Ji-Yong Chung (Department of Medical Device Engineering and Management, Yonsei University College of Medicine) ;
  • You Rim Kim (Department of Medical Device Engineering and Management, Yonsei University College of Medicine) ;
  • Wonseuk Jang (Department of Medical Device Engineering and Management, Yonsei University College of Medicine)
  • 정지용 (연세대학교 대학원 의료기기산업학과) ;
  • 김유림 (연세대학교 대학원 의료기기산업학과) ;
  • 장원석 (연세대학교 대학원 의료기기산업학과)
  • Received : 2023.10.26
  • Accepted : 2023.11.07
  • Published : 2023.12.31
Download PDF
⟨ Previous Next ⟩

Abstract

According to the COVID-19, development of various medical software based on IoT(Internet of Things) was accelerated. Especially, interest in a central software system that can remotely monitor and control ventilators is increasing to solve problems related to the continuous increase in severe COVID-19 patients. Since medical device software is closely related to human life, this study aims to develop central monitoring system that can remotely monitor and control multiple ventilators in compliance with medical device software development standards and to verify performance of system. In addition, to ensure the safety and reliability of this central monitoring system, this study also specifies risk management requirements that can identify hazardous situations and evaluate potential hazards and confirms the implementation of cybersecurity to protect against potential cyber threats, which can have serious consequences for patient safety. As a result, we obtained medical device software manufacturing certificates from MFDS(Ministry of Food and Drug Safety) through technical documents about performance verification, risk management and cybersecurity application.The purpose of this study is to conduct a usability assessment to ensure that ergonomic design has been applied so that the ventilator central monitoring system can improve user satisfaction, efficiency, and safety. The rapid spread of COVID-19, which began in 2019, caused significant damage global medical system. In this situation, the need for a system to monitor multiple patients with ventilators was highlighted as a solution for various problems. Since medical device software is closely related to human life, ensuring their safety and satisfaction is important before their actual deployment in the field. In this study, a total of 21 participants consisting of respiratory staffs conducted usability test according to the use scenarios in the simulated use environment. Nine use scenarios were conducted to derive an average task success rate and opinions on user interface were collected through five-point Likert scale satisfaction evaluation and questionnaire. Participants conducted a total of nine use scenario tasks with an average success rate of 93% and five-point Likert scale satisfaction survey showed a high satisfaction result of 4.7 points on average. Users evaluated that the device would be useful for effectively managing multiple patients with ventilators. However, improvements are required for interfaces associated with task that do not exceed the threshold for task success rate. In addition, even medical devices with sufficient safety and efficiency cannot guarantee absolute safety, so it is suggested to continuously evaluate user feedback even after introducing them to the actual site.

Keywords

Acknowledgement

연구는 2020년 범부처의 연구개발비(202011B26)로 수행되었으며 이에 감사드립니다.

References

  1. Thobaity AA, Alshammari F. Nurses on the frontline against the COVID-19 pandemic: an integrative review. Dubai Medical Journal. 2020;3(3):87-92. https://doi.org/10.1159/000509361
  2. Chung JY, Kim YR, Jang WS. Software Risk Management and Cyber Security for Development of Integrated System Remotely Monitoring and Controlling Ventilators. Journal of Biomedical Engineering Research. 2023;44(2):99-108. https://doi.org/10.9718/JBER.2023.44.2.99
  3. MFDS. Guideline on the Review and Approval of Digital Therapeutics(DTx). Cheongju: Ministry of Food and Drug Safety;2020.
  4. Resnic FS, Matheny ME. Medical devices in the real world. New England Journal of Medicine. 2018;378(7):595-597. https://doi.org/10.1056/NEJMp1712001
  5. Jandoo T. WHO guidance for digital health : What it means for researchers. Digital health. 2020;6:1-4. https://doi.org/10.1177/2055207619898984
  6. Schmettow M, Schnittker R, Schraagen JM. An extended protocol for usability validation of medical devices: Research design and reference model. Journal of biomedical informatics. 2017;69:99-114. https://doi.org/10.1016/j.jbi.2017.03.010
  7. McCarthy AD, Moody L, Reeves ML, Healey TJ, Good T, Sproson L, Adebajo A, Tindale W, Nair KPS. Usability engineering in practice: developing an intervention for post-stroke therapy during a global pandemic. Journal of Medical Engineering & Technology. 2022;46(6):433-447. https://doi.org/10.1080/03091902.2022.2089257
  8. Lee YJ, Hong JP, Jung KT. User Experience Evaluation Method using Multi-Criteria Decision Making. Journal of the Ergonomics Society of Korea. 2022;41(2):141-150. https://doi.org/10.5143/JESK.2022.41.2.141
  9. Wronikowska MW, Malycha J, Morgan LJ, Westgate V, Petrinic T, Young JD, Watkinson PJ. Systematic review of applied usability metrics within usability evaluation methods for hospital electronic healthcare record systems: Metrics and Evaluation Methods for eHealth Systems. Journal of Evaluation in Clinical Pracitce, 2021;27(6):1403-1416. https://doi.org/10.1111/jep.13582
  10. IEC. IEC 62366-1 : 2015, Medical devices-Part 1 : Application of usability engineering to medical devices. Geneva: International Electrotechnical Commission; 2015.
  11. IEC. IEC TR 62366-2, Medical devices-Part 2 : Guidance on the application of usability engineering to medical devices. Geneva: International Electrotechnical Commission; 2016.
  12. Sauer J, Sonderegger A, Heyden K, Biller J, Klotz J, Uebelbacher A. Extra-laboratorial usability tests: An emirical comparison of remote and classical field testing with lab testing. Applied ergonomics. 2019;74:85-96. https://doi.org/10.1016/j.apergo.2018.08.011
  13. ANSI/AAMI HE 75 : Human Factors Engineering-Design of Medical Devices. Association for the Advancement of Medical Instrumentation; 2009.
  14. Haig F, Medeiros ACB, Chitty K, Slack M. Usability assessment of Versius, a new robot-assisted surgical device for use in minimal access surgery. BMJ Surgery, Interventions, & Health Technologies. 2020;2(1).
  15. Rochat J, Ehrler F, Siebert JN, Ricci A, Ruiz VG, Lovis C. Usability Testing of a Patient-Centered Mobile Health App for Supporting and Guiding the Pediatric Emergency Department Patient Journey: Mixed Methods Study. JMIR Pediatrics and Parenting. 2022;5(1):e25540.
  16. Bird M, Carter N, Lim A, Kazmie N, Fajardo C, Reaume S, Mcgillion MH. A novel hospital-to-home system for children with medical complexities: usability testing study. JMIR Formative Research. 2022;6(8):e34572.
  17. Hinricher N, Konig S, Schroer C, Backhaus C. Effect of virtual reality and test environment on user experience, usability, and mental workload in the evaluation of a blood pressure monitor. Frontiers in Virtual Reality. 2023;4:115119.
  18. Templier F, Miroux P, Dolveck F, Descatha A, Goddet NS, Jeleff C, Fletcher D. Evaluation of the ventilator-user interface of 2 new advanced compact transport ventilators. Respiratory care. 2007;52(12):1701-1709.
  19. South L, Saffo D, Vitek O, Dunne C, Borkin MA. Effective use of likert scales in visualization evaluations: a systematic review. Computer Graphics Forum 2022;41(3):43-55. https://doi.org/10.1111/cgf.14521
  20. Shin JH, Lee H. Optimal Usability Test Procedure Generation for Medical Devices. Healthcare. 2023;11(3):296.
  21. ISO. ISO 14971, Medical devices - Application of risk management to medical devices. Geneva: International Organization for Standardization; 2019.
  22. Coronavirus resource center. Johns Hopkins University & Medicine. 2021. https://coronavirus.jhu.edu/map.html accessed May. 23, 2023.
  23. Ha DS, Im YJ, Jung ES. Developing of a New Touch Interface for PC Remote Control Application. Journal of the Ergonomics Society of Korea. 2021;40(4):235-248. https://doi.org/10.5143/JESK.2021.40.4.235
  24. Umair M. Impact of COVID-19 on IoT adoption in healthcare, smart homes, smart buildings, smart cities, transportation and industrial IoT. Sensors. 2021;21(11):3838.
  25. Scott JB, Vaux LD, Dills C, Strickland SL. Mechanical ventilation alarms and alarm fatigue. Respiratory Care. 2019;64(10):1308-1313. https://doi.org/10.4187/respcare.06878