Journal of Korean Society of Industrial and Systems Engineering (산업경영시스템학회지)

Society of Korea Industrial and System Engineering (한국산업경영시스템학회)
권호 표지

Attainment Index-based Relative Evaluation Method for R&D Programs with Heterogeneous Objectives

이질적 목적을 지닌 R&D 사업들을 위한 달성지수 기반의 상대적 평가기법

  • Jung, Uk (Dept. of Management, Dongguk University) ;
  • Yim, Seong-Min (Korean Institute of Science and Technology Evaluation and Planning) ;
  • Kim, Yun-Jong (Korean Institute of Science and Technology Evaluation and Planning) ;
  • Jeong, Sang-Ki (Korean Institute of Science and Technology Evaluation and Planning)
  • 정욱 (동국대학교 경영학과) ;
  • 임성민 (한국과학기술기획평가원) ;
  • 김윤종 (한국과학기술기획평가원) ;
  • 정산기 (한국과학기술기획평가원)
  • Published : 2009.06.30
Download PDF
⟨ Previous Next ⟩

Abstract

National R&D programs play an important role in the development of a country in this age of the knowledge economy. Since many numbers of R&D programs compete for limited resources such as national R&D budget, the R&D program evaluation problem is a challenging decision-making problem faced by decision makers that deal with R&D management. In this sense, DEA(Data Envelopment Analysis) has been regarded as one of the most widely accepted methods to measure the relative efficiency of productivity of R&D programs. DEA is a methodology to measure and to evaluate the relative efficiency of a homogeneous set of decision-making units(DMUs) in a process which uses multiple inputs to produce multiple outputs. However, the sample of the R&D programs could consist of two or more naturally occurring subsets, thus exhibiting clear signs of heterogeneity such as different objectives. In such situations, the fairness of DEA is limited, for the nature of the relative efficiency of a DMU is likely to be influenced by its membership in a particular subset of the sample. In this study, we propose a methodology AI-DEA(attainment index DEA) allowing for reflecting decision maker's subjective judgement on difference among different subsets of R&D programs which have heterogeneous objectives. This methodology combines AHP and Delphi in order to decide the attainmnet index of each DMU for each outputs, and apply them to DEA model. We illustrate the proposed approach with a pilot evaluation of 13 programs involving 6 different subsets of Korean National R&D programs and compares the results of the original DEA model and AI-DEA model.

Keywords

References

  1. 과학기술기본법(일부개정 2008. 6. 5 법률 제9089호)
  2. 교육과학기술부 “국가연구개발-국가연구개발사업평가”, http://www.most.go.kr, 2008
  3. 국가과학기술위원회, 2007년도 국가연구개발사업 조사.분석 보고서, 2007
  4. 기획재정부; 2008년도 국가연구개발사업 상위평가지침, 2008
  5. 김윤종; “국가 R&D 성과평가 개요 및 제도개선 방안”, 한국행정학회 추계학술대회, 2007
  6. 손소영, 주용규; “분류모형과 DEA를 이용한 두뇌한국(BK) 21 사업단 효율성 분석”, IE Interfaces, 17(3) : 249-260, 2004
  7. 임호순, 유석천, 김연성; "연구개발사업의 평가 및 선정을 위한 DEA/AHP 통합모형에 관한 연구", 한국경영과학회지, 24(4) : 1-12, 1999
  8. 조용곤, 조근태; “Delphi와 AHP를 이용한 생명공학분야 미래유망기술의 R&D전략 수립”, 한국경영과학회 2004년 춘계학술대회논문집 : 183-186, 2004
  9. Banker, R. D. and Morey, R. C., "The use of categorical variables in data envelopment analysis," Management Science, 32(12) : 1613-1627, 1986 https://doi.org/10.1287/mnsc.32.12.1613
  10. Belton, V. and Vickers, S. P.; "Demystifying DEA : A visual interactive approach," Journal of Operations Research Society, 44(9) : 883-896, 1993
  11. Chang, H.; "Determinant of hospital efficiency: The case of central government owned hospitals in Taiwan, Omega," International Journal of Management Science, 26(2) : 307-317, 1998 https://doi.org/10.1016/S0305-0483(98)00014-0
  12. Charnes, A., Cooper, W. W., and Rhodes, E.; "Measuring the Efficiency of Decision Making Units," European Journal of Operational Research, 2(2) : 429-444, 1978 https://doi.org/10.1016/0377-2217(78)90138-8
  13. Eilat, H., Golany, B., and Shtub, A.; "R&D projects evaluation: An integrated DEA and balanced scorecard approach, Omega," International Journal of Management Science, 36(5) : 895-912, 2008 https://doi.org/10.1016/j.omega.2006.05.002
  14. Kao, C. Hung, H. T.; "Efficiency analysis of university departments: An empirical study," Omega, International Journal of Management Science, 36(4) : 653-664, 2008 https://doi.org/10.1016/j.omega.2006.02.003
  15. Lee, H. Park, Y., and Chio, H., "Comparative evaluation of performance of national R&D programs with heterogeneous objectives: A DEA approach," European Journal of Operational Research, In press, Available online 26 June 2008
  16. Meng, W., Hu, Z. H., and Liu, W. B.; "Efficiency evaluaiton of basic research in China," Scientometrics, 69 : 85-101, 2006 https://doi.org/10.1007/s11192-006-0140-y
  17. Meng, W., Zhang, D., Qi, L., and Liu, W.; "Twolevel DEA approaches in research evaluation," OMEGA, 36 : 950-957, 2008 https://doi.org/10.1016/j.omega.2007.12.005
  18. O'Neill, L, Rauner, M., Heidenberger, K., and Kraus, M.; "A cross-national comparison and taxonomy of DEA-based hospital efficiency studies," Socio-Economic Planning Sciences, 42(3) : 158-189, 2008 https://doi.org/10.1016/j.seps.2007.03.001
  19. Oral, Kettani, and Lang; "A methodology for collective evaluation and Selection of industrial R&D projects," Management Science, 37(7) : 871-885, 1991 https://doi.org/10.1287/mnsc.37.7.871
  20. Saaty, T., "Priority Setting in Complex Problem," IEEE Transactions on Engineering Management, 30(3) : 140-155, 1983
  21. Schaffnit, C., Rosen, D., and Paradi, J. C.; "Best practice analysis of bank branches : An application of DEA in a large Canadian bank," European Journal of Operational Research, 98 : 269-289, 1997 https://doi.org/10.1016/S0377-2217(96)00347-5
  22. Sowlati, T., Paradi, J. C., and Suld, C., "Information systems project prioritization using data envelopment analysis," Mathematival and Computer Modelling, 41 (11-12) : 1279-1298, 2005 https://doi.org/10.1016/j.mcm.2004.08.010
  23. Thompson, R. G., Langemerer, L. N., Lee, C. T., Lee, E., and Thrall, R. M.; "The role of multiplier bounds in efficiency analysis with application to Kansas Farming," Journal of Econometrics, 46 : 93-108, 1990 https://doi.org/10.1016/0304-4076(90)90049-Y
  24. Zhu, J.; Quantitative models for performance evaluation and benchmarking, Springer's International Series, 2003