Journal of the Korea Academia-Industrial cooperation Society (한국산학기술학회논문지)

The Korea Academia-Industrial cooperation Society (한국산학기술학회)
권호 표지
DOI QR코드

DOI QR Code

Development of Methodology for Evaluation Performance Model of Information Systems

정보시스템 성과 평가 모형 방법론 개발에 관한 연구

  • Kim, Changkyu (Department of business and administration, Kwnagwoon University) ;
  • Park, Wonhee (Department of Internet Information, Osan University)
  • Received : 2016.07.13
  • Accepted : 2016.08.11
  • Published : 2016.08.31
Download PDF
⟨ Previous Next ⟩

Abstract

In the information systems literature from Korea, there has not been much research on formative constructs. It is crucial to establish a proper relationship between constructs and indicators. In other words, it is fundamental to specify reflective or formative constructs in evaluating performance as closely as possible to reality, and in testing the appropriateness of a proper causal model. One purpose of this study is that, through a comprehensive literature review, reflective and formative indicators are accurately understood, and a proper specification and development methodology is applied to the information system evaluation field. In addition, this study provides a useful guideline for developing formative indicators for performance evaluation of informatization programs. The following activities were undertaken to achieve the aforementioned purposes. First, the basic theories and preceding study models on successful factors of informatization programs and performance evaluations were reviewed, and a comprehensive interdisciplinary literature review was conducted to better understand the formative constructs. Lastly, we provide a construct for performance evaluation of informatization programs and evaluation indicators, as well as guidelines for specifying them. Therefore, by systematically specifying proper constructs, future domestic researchers can develop better constructs for performance evaluation of informatization programs.

국내 정보시스템 문헌에서는 형성적 구성변수에 대한 연구가 거의 없다. 따라서 올바른 구성변수와 측정변수의 관계, 즉 반영적 혹은 형성적 구성변수의 설정은 가장 현실에 가까운 성과 평가와 올바른 인과모형을 검정하는 데 필수적이다. 본 연구 목적은 포괄적인 문헌 고찰로 반영지표와 형성지표에 대해 정확히 이해하고 올바른 구성 및 개발 방법론을 정보시스템 평가 분야에 적용하는 것이다. 그리고 정보시스템 성과 평가에 대한 형성적 지표를 개발할 수 있는 올바른 지침을 제시하는 것이다. 이와 같은 목적을 달성하고자 다음과 같은 연구과정을 수행하였다. 먼저, 정보화사업 성과 요인과 성과 평가에 관한 기저 이론과 선행 연구 모형을 검토하였다. 그리고 정보화사업 성과 평가 구성변수와 평가지표 개발 및 타당성 검증을 위한 지침(Guideline)을 통하여 정보화사업 성과 평가에 대한 올바른 형성적 구성변수 및 반영적 구성변수를 식별하였다. 이에 따라, 체계적으로 구성변수를 올바르게 식별 및 분류하여, 향후 연구자가 정보화사업 성과 평가 구성변수의 올바른 구성변수를 개발할 수 있도록 하였다.

Keywords

References

  1. Straub, D. W., "Validating Instruments in MIS Research," MIS Quarterly, vol. 13, no. 2, pp. 147-169, 1989. DOI: http://dx.doi.org/10.2307/248922
  2. Gefen, D., D. W. Straub, and M. C. Boudreau, Structural Equation Modeling and Regression: Guidelines for Research Practice, 2000.
  3. Kim, G., B. Shin, and V. Grover, "Investigating Two Contradictory Views of Formative Measurement in Information Systems Research," MIS Quarterly, vol. 34, no. 2, pp. 345-365, 2010. https://doi.org/10.2307/20721431
  4. Chin, W. W., and P. A. Todd, "On the use, Usefulness, and Ease of use of Structural Equation Modeling in MIS Research: A Note of Caution," MIS Quarterly, pp. 237-246, 1995. DOI: http://dx.doi.org/10.2307/249690
  5. Petter, S., D. Straub, and A. Rai, "Specifying Formative Constructs in Information Systems Research," Management Information Systems Quarterly, vol. 31, no. 4, pp. 623-656, 2007. https://doi.org/10.2307/25148814
  6. Cenfetelli, R. T., and G. Bassellier, "Interpretation of Formative Measurement in Information Systems Research," Mis Quarterly, vol. 33, no. 4, pp. 689-707, 2009. https://doi.org/10.2307/20650323
  7. Swanson, E. Burton, "Management Information Systems: Appreciation and Involvement", Management Science, vol. 21, no. 2, pp. 178-188, 1974. DOI: http://dx.doi.org/10.1287/mnsc.21.2.178
  8. Boland, R., "The Process and Product of System Design," Management Science, vol. 24, no. 9, pp. 887-898, 1978. DOI: http://dx.doi.org/10.1287/mnsc.24.9.887
  9. Borovits, I., and Ein-Dor, P., "Cost/Utilization: A Measure of System Performance", Communications of the ACM, vol. 20, no. 3, pp. 185-191, 1977. DOI: http://dx.doi.org/10.1145/359436.359452
  10. Borovits, I., and Neumann, S., Computer Systems Performance Evaluation, D. C. Heath and Co., Lexington, MA, 1979.
  11. Chandler, J. S., "A Multiple Criteria Approach for Evaluating Information Systems," MIS Quarterly, vol. 6, no. 1, pp. 61-74, 1982. DOI: http://dx.doi.org/10.2307/248755
  12. Li, E. Y., "Perceived Importance of Information Systems Success Factors: A Meta Analysis of Group Differences. Information and Management," vol. 32, no. 1, pp. 15-28, 1997. DOI: http://dx.doi.org/10.1016/S0378-7206(97)00005-0
  13. Bailey, J. E., and S. W. Pearson, "Development of A Tool for Measuring and Analyzing Computer User Satisfaction," Management Science, vol. 29, no. 1, pp.530-545, 1983. DOI: http://dx.doi.org/10.1287/mnsc.29.5.530
  14. Seddon, P. B., and M-Y Kiew, "A Partial Test and Development of the DeLone and McLean Model of IS Success" (1994). Proceedings of the International Conference on Information Systems, Vancouver, Canada (ICIS 94), pp. 99-110, 1994.
  15. Seddon, P. B., "A Respecification and Extension of the DeLone and McLean Model of IS Success" Information Systems Research, vol. 8, no. 3, pp. 240-253, 1997. DOI: http://dx.doi.org/10.1287/isre.8.3.240
  16. Pitt, L. F., and R. T. Watson, "Service Quality: A Measure of Information Systems Effectiveness," MIS Quarterly, vol. 19, no. 2, pp. 173-187, 1995. DOI: http://dx.doi.org/10.2307/249687
  17. Pitt, L., Watson, R., and Kavan, C., "Measuring Information Systems Service Quality: Concerns for a Complete Canvas," MIS Quarterly, vol. 21, no. 2, pp. 167-182, 1997. DOI: http://dx.doi.org/10.2307/249420
  18. Kettinger, W. J., and Lee, C. C., "Perceived Service Quality and User Satisfaction with the Information Services Function," Decision Sciences, vol. 25, no. 5, pp. 737-766, 1994. DOI: http://dx.doi.org/10.1111/j.1540-5915.1994.tb01868.x
  19. Kettinger, W. J., and Lee, C. C., "Pragmatic perspectives on the measurement of information systems service quality," MIS Quarterly, vol. 21, no. 2, pp. 223-240, 1997. DOI: http://dx.doi.org/10.2307/249421
  20. DeLone, W. H., and McLean, E. R., "Information Systems Success: The Quest for the Dependent Variable," Information Systems Research, vol. 3, no. 1, pp. 60-95, 1992. DOI: http://dx.doi.org/10.1287/isre.3.1.60
  21. Shannon, C. E., and Weaver, W., The mathematical theory of information, Urbana: University of Illinois Press, 1949.
  22. DeLone, W. H., and McLean, E. R., "Information systems success revisited," In Proceedings of the 35th Hawaii International Conference on System Sciences (SPRAGUE JR RH, Ed), pp. 3057-3067, IEEE Computer Society, Hawaii, US, 2002. DOI: http://dx.doi.org/10.1109/HICSS.2002.994345
  23. DeLone, W. H., and McLean, E. R., "The DeLone and McLean Model of Information Systems Success: A Ten-Year Update," Journal of Management Information Systems, vol. 19, no. 4, pp. 9-30, 2003. https://doi.org/10.1080/07421222.2003.11045748
  24. MacCallum, R. C., and M. W. Browne. "The use of Causal Indicators in Covariance Structure Models: Some Practical Issues," Psychological Bulletin, vol. 114, no. 3, pp. 533-541, 1993. DOI: http://dx.doi.org/10.1037/0033-2909.114.3.533
  25. Jarvis, C. B., S. B. MacKenzie, and P. M. Podsakoff, "A Critical Review of Construct Indicators and Measurement Model Misspecification in Marketing and Consumer Research," Journal of Consumer Research, vol. 30, no. 2, pp. 199-218, 2003. DOI: http://dx.doi.org/10.1086/376806
  26. Bollen, K., and R. Lennox, "Conventional Wisdom on Measurement: A Structural Equation Perspective," Psychological Bulletin, vol. 110, no. 2, pp. 305-314, 1991. DOI: http://dx.doi.org/10.1037/0033-2909.110.2.305
  27. Howell, R., E. Breivik, James B. Wilcox, "Reconsidering formative measurement," Psychological methods, vol. 12, no. 2, pp. 205-218, 2007. DOI: http://dx.doi.org/10.1037/1082-989X.12.2.205
  28. Burt, Ronald S., "Interpretational Confounding of Unobserved Variables in Structural Equation Models," Sociological Methods and Research, vol. 5, pp. 3-52, 1976. DOI: http://dx.doi.org/10.1177/004912417600500101
  29. Bollen, K. A., "Interpretational Confounding Is Due to Misspecification, Not to Type of Indicator: Comment on Howll, Breivik, and Wilcox," Psychological Methods, vol. 12, no. 2, pp. 219-228, 2007. DOI: http://dx.doi.org/10.1037/1082-989X.12.2.219
  30. Kim, Sanghoon and Kim, Changkyu, "Development of Formative Constructs and Measurements for Performance Evaluation of Information Systems, Journal of the Korea society of IT services, vol. 11, no. 4, pp.135-152, 2012. https://doi.org/10.9716/KITS.2012.11.4.135
  31. KIMI (Korea Information Management Institute for SMEs), A Study on the Franeworks of Performance Evaluation on the Informatization Support policy for SMEs, 11. 2005.
  32. Bagozzi, R. P., "Measurement and Meaning in Information Systems and Organizational Research: Methodological and Philosophical Foundations," MIS Quarterly, vol. 35, no. 2, pp. 261-292, 2011. https://doi.org/10.2307/23044044