The Journal of Society for e-Business Studies (한국전자거래학회지)

Society for e-Business Studies (한국전자거래학회)
권호 표지
DOI QR코드

DOI QR Code

Securing Objectivity of Qualitative Assessment Results using Ordered Probit Model

순서형 프로빗 모델을 활용한 정성적 평가 결과의 객관성 확보방안

  • Received : 2022.02.03
  • Accepted : 2022.02.17
  • Published : 2022.02.28
Download PDF
⟨ Previous Next ⟩

Abstract

In the service sectors, the qualitative evaluation method in the form of a survey is widely used as a major assessment tool to evaluate the quality of service. However, the results obtained from a survey can involve the subjective judgment of the respondent. In this study, we propose a method to secure objectivity by excluding subjectivity that may be included in the qualitative evaluation results. In particular, we deal with a situation where the same type of qualitative evaluation tool is used repeatedly by several service providers. To this end, by utilizing both the Ordered Probit model and third-party evaluation results, we determine whether subjectivity is involved in the results. After correcting subjectivity, the final results are obtained through statistical analysis. The application analyzed in this study is the medical service area. With the actual evaluation results supplied by the service providers, we explain how objectivity can be secured from the assessment data by applying our proposed approach.

서비스 산업에서 널리 사용되는 설문조사 형태의 정성적 평가방법은 서비스 품질을 평가하는 주요한 평가 수단으로 활용되는데, 이때 설문조사 결과에 응답자의 주관적 판단이 개입될 수 있는 여지가 존재한다. 본 연구에서는 이러한 정성적 평가 결과에 포함될 수 있는 주관적 판단을 최대한 배제하고 객관성을 확보하기 위한 방안을 제시한다. 특히 같은 종류의 정성적 평가도구가 여러 사람에 의해 반복적으로 사용되는 상황에서의 객관성 확보 방안에 대해 연구하였다. 이를 위해, Ordered Probit 모델 및 제3자 평가 결과를 함께 활용하여 주관성 개입 여부를 판단하고, 주관성을 보정한 후, 최종 결과에 대해 통계적으로 검증하는 절차를 수행하였다. 본 연구에서 분석한 어플리케이션은 의료서비스 분야이며, 특히 특정 의료서비스 제공 환경에서 해당 서비스 제공자가 자가 진단한 실제 평가 결과를 본 연구에서 제안하는 방법론을 통해 어떻게 객관성이 확보될 수 있는지 구체적으로 설명하였다.

Keywords

Acknowledgement

This work was supported by 2019 Hongik University Research Fund.

References

  1. Aitchison, J. and Silvey, S. D., "The generalization of probit analysis to the case of multiple responses," Biometrika, Vol. 44, No. 1/2, pp. 131-40, 1957. https://doi.org/10.1093/biomet/44.1-2.131
  2. Aldrich, J. H. and Nelson, F. D., Linear Probability, Logit, and Probit Models, No. 45, Sage, 1984.
  3. Folstein, M. F., Robins, L. N., and Helzer, J. E., "The mini-mental state examination," Archives of General Psychiatry, Vol. 40, No. 7, pp. 812-812, 1983. https://doi.org/10.1001/archpsyc.1983.01790060110016
  4. Ju, M., "Probit and ordered probit analysis and its application," Journal of Governmental Studies, Vol. 6, No. 1, pp. 24-49, 2000.
  5. Kang, S., "How to estimate inter-rater reliability?," Korean Society For Measurement And Evaluation In Physical Education And Sports Science, Vol. 13, No. 1, pp. 1-8, 2011.
  6. Lee, G., Kang, K., and Rho, J., "Development of bicycle level of service model from the user's perspectiveusing ordered probit model," Journal of Korean Institute of Intelligent Transport Systems, Vol 8, No. 2, pp. 108-117, 2009.
  7. Mckelvey, R. D. and Zavoina, W., "A statistical model for the analysis of ordinal level dependent variables," Journal of Mathematical Sociology, Vol. 4, No. 1, pp. 103-120, 1975. https://doi.org/10.1080/0022250X.1975.9989847
  8. Myers, J. L., Well, A. D., and Lorch Jr, R. F., Research Design and Statistical Analysis, Routledge, 2013.
  9. Park, J. and Kim, S., "The significance test on the AHP-based alternative evaluation: An application of non-parametric statistical method," The Journal of Society for e-Business Studies, Vol. 22, No. 1, pp. 15-35, 2017.
  10. Reeve, B. B. and Fayers, P., "Applying item response theory modeling for evaluating questionnaire item and scale properties," Assessing quality of life in clinical trials: Methods of practice, Vol. 2, pp. 55-73, 2005.
  11. Van der Linden, W. J. and Hambleton, R. K., Handbook of modern item response theory, Springer Science & Business Media, 2013.
  12. Wilcoxon, F., Individual Comparisons by Ranking Methods, pp. 196-202, Breakthroughs in statistics, Springer, New York, 1992.