Journal of Korean Institute of Industrial Engineers (대한산업공학회지)

Korean Institute of Industrial Engineers (대한산업공학회)
권호 표지
DOI QR코드

DOI QR Code

Identification of DEA Determinant Input-Output Variables : an Illustration for Evaluating the Efficiency of Government-Sponsored R&D Projects

DEA 효율성을 결정하는 입력-출력변수 식별 : 정부지원 R&D 과제 효율성 평가를 위한 실례

  • Park, Sungmin (Department of Business Administration, Baekseok University)
  • Received : 2013.06.01
  • Accepted : 2013.09.02
  • Published : 2014.02.15
Download PDF
⟨ Previous Next ⟩

Abstract

In this study, determinant input-output variables are identified for calculating Data Envelopment Analysis (DEA) efficiency scores relating to evaluating the efficiency of government-sponsored research and development (R&D) projects. In particular, this study proposes a systematic framework of design and analysis of experiments, called "all possible DEAs", for pinpointing DEA determinant input-output variables. In addition to correlation analyses, two modified measures of time series analysis are developed in order to check the similarities between a DEA complete data structure (CDS) versus the rest of incomplete data structures (IDSs). In this empirical analysis, a few DEA determinant input-output variables are found to be associated with a typical public R&D performance evaluation logic model, especially oriented to a mid- and long-term performance perspective. Among four variables, only two determinants are identified : "R&D manpower" ($x_2$) and "Sales revenue" ($y_1$). However, it should be pointed out that the input variable "R&D funds" ($x_1$) is insignificant for calculating DEA efficiency score even if it is a critical input for measuring efficiency of a government-sonsored R&D project from a practical point of view a priori. In this context, if practitioners' top priority is to see the efficiency between "R&D funds" ($x_1$) and "Sales revenue" ($y_1$), the DEA efficiency score cannot properly meet their expectations. Therefore, meticulous attention is required when using the DEA application for public R&D performance evaluation, considering that discrepancies can occur between practitioners' expectations and DEA efficiency scores.

Keywords

References

  1. Abramo, G., D'Angelo, C. A., and Pugini, F. (2008), The Measurement of Italian Universities' Research Productivity by a Non Parametric-bibliometric Methodology, Scientometrics, 76(2), 225-244. https://doi.org/10.1007/s11192-007-1942-2
  2. Allen, R., Athanassopoulos, A., Dyson, R. G., and Thanassoulis, E. (1997), Weights Restrictions and Value Judgements in Data Envelopment Analysis, Annals of Operations Research, 73(1), 13-34. https://doi.org/10.1023/A:1018968909638
  3. Asmild, M., Paradib, J. C., Reesec, D. N., and Tamb, F. (2007), Measuring Overall Efficiency and Effectiveness Using DEA, European Journal of Operational Research, 178 (1), 305-321. https://doi.org/10.1016/j.ejor.2006.01.014
  4. Banker, R. D., Charnes, A. and Cooper, W. W. (1984), Some Models for Estimating Technical and Scale Inefficiencies in Data Envelopment Analysis, Management Science, 30(9), 1078-1092. https://doi.org/10.1287/mnsc.30.9.1078
  5. Banker, R. D., Conrad, R. F., and Strauss, R. P. (1986), A Comparative Application of Data Envelopment Analysis and Translog Methods : An Illustrative Study of Hospital Production, Management Science, 32(1), 30-44. https://doi.org/10.1287/mnsc.32.1.30
  6. Banker, R. D. and Morey, R. (1986a), Efficiency Analysis for Exogenously Fixed Inputs and Outputs, Operations Research, 34(4), 513-521. https://doi.org/10.1287/opre.34.4.513
  7. Banker, R. D. and Morey, R. (1986b), The Use of Categorical Variables in Data Envelopment Analysis, Management Science, 32(12), 1613-1627. https://doi.org/10.1287/mnsc.32.12.1613
  8. Berenson, M. L., Levine, D. M., and Krehbiel, T. C. (2012), Basic Business Statistics, 12th ed., Boston MA : Pearson Education Limited.
  9. Bessent, A., Bessent, W., Kennington, J., and Reagan, B. (1982), An Application of Mathematical Programming to Assess Productivity in the Houston Independent School District, Management Science, 28(12), 1355-1367. https://doi.org/10.1287/mnsc.28.12.1355
  10. Bickman, L. (1987), The Functions of Program Theory, Special Issue : Using Program Theory in Evaluation, New Directions for Program Evaluation, 1987(33), 5-18.
  11. Bitman, W. R. and Sharif, N. (2008), A Conceptual Framework for Ranking R&D Projects, IEEE Transactions on Engineering Management, 55(2), 267-278.
  12. Callen, J. L. (1991), Data Envelopment Analysis : Partial Survey and Applications for Management Accounting, Journal of Management Accounting Research, 3, 35-56.
  13. Charnes, A., Clark, C. T., Cooper, W. W., and Golany, B. (1985), A Developmental Study of Data Envelopment Analysis in Measuring the Efficiency of Maintenance Units in the US Air Forces, In Thompson, R. G. and Thrall, R. M. (Ed.), Annals of Operations Research, 2(1), 95-112.
  14. Charnes, A. and Cooper, W. W. (1980), Auditing and Accounting for Program Efficiency and Management Efficiency in Not-for-profit Entities, Accounting, Organizations and Society, 5(1), 87-107. https://doi.org/10.1016/0361-3682(80)90025-2
  15. Charnes, A., Cooper, W. W., Huang, Z. M. and Sun, D. B. (1990), Polyhedral Cone-ratio DEA Models with an Illustrative Application to Large Commercial Banks, Journal of Econometrics, 46(1/2), 73-91. https://doi.org/10.1016/0304-4076(90)90048-X
  16. Charnes, A., Cooper, W. W. and Rhodes, E. (1978), Measuring the Efficiency of Decision Making Units, European Journal of Operational Research, 2(6), 429-444. https://doi.org/10.1016/0377-2217(78)90138-8
  17. Charnes, A., Cooper, W. W., and Rhodes, E. (1981), Evaluating Program and Managerial Efficiency : An Application of Data Envelopment Analysis to Program Follow Through, Management Science, 27(6), 668-697. https://doi.org/10.1287/mnsc.27.6.668
  18. Chiesa, V. and Masella, C. (1996), Searching for an Effective Measure of R&D Performance, Management Decision, 34(7), 49-57.
  19. Cooper, W. W., Seiford, L. M., and Tone, K. (2007), Data Envelopment Analysis : A Comprehensive Text With Models, Applications, References and DEA-Solver Software, 2nd ed., New York NY : Springer.
  20. Cooper, W. W., Seiford, L. M., and Zhu, J. (2004), Handbook on Data Envelopment Analysis, Boston MA : Springer.
  21. DEA-Solver-Pro (2012), Professional Version 8.0 with Excel 2010 on 64 bit Windows 7, Holmdel NJ : SAITECH, Inc.
  22. Farris, J. A., Groesbeck, R. L., Aken, E. M. V., and Letens, G. (2006), Evaluating the Relative Performance of Engineering Design Projects : A Case Study Using Data Envelopment Analysis, IEEE Transactions on Engineering Management, 55(3), 471-482.
  23. Government Performance Results Act (GPRA) (1993), U.S. Office of Management and Budget, Available online at : http://www.whitehouse.gov/omb/mgmt-gpra/gplaw2m(accessed 1 October 2012).
  24. Guan, J. and Chen, K. (2010), Modeling Macro-R&D Production Frontier Performance : An Application to Chinese Province-level R&D, Scientometrics, 82(1), 165-173. https://doi.org/10.1007/s11192-009-0030-1
  25. Hsu, F. M. and Hsueh, C. C. (2009), Measuring Relative Efficiency of Government-sponsored R&D Projects : A Three-stage Approach, Evaluation and Program Planning, 32(2), 178-186. https://doi.org/10.1016/j.evalprogplan.2008.10.005
  26. Lee, H., Park, Y. and Choi, H. (2009), Comparative Evaluation of Performance of National R&D Programs with Heterogeneous Objectives : A DEA Approach, European Journal of Operational Research, 196(3), 847-855. https://doi.org/10.1016/j.ejor.2008.06.016
  27. McLaughlin, J. A. and Jordan, G. B. (1999), Logic Models : A Tool for Telling Your Program's Performance Story, Evaluation and Program Planning, 22(1), 65-72. https://doi.org/10.1016/S0149-7189(98)00042-1
  28. Meng, W., Hu, Z. H., and Liu, W. B. (2006), Efficiency Evaluation of Basic Research in China, Scientometrics, 69(1), 85-101. https://doi.org/10.1007/s11192-006-0140-y
  29. Ministry of Knowledge and Economy (MKE)․National IT Industry Promotion Agency (NIPA) (2012), Performance Analysis on Information and Communication Promotion Fund (VII), Korea : MKE․NIPA.
  30. Ministry of Science and Technology (MST)․Korea Institute of Science and Technology Evaluation and Planning (KISTEP) (2007), A Study of the Methodology for Follow-up Evaluation of National R&D Programs, Korea : MST․KISTEP.
  31. MinitabR (2005), MinitabR Release 14.20 StatGuide, State College PA : Minitab Inc.
  32. Montgomery, D. C., Peck, E. A., and Vining, G. G. (2001), Introduction to Linear Regression Analysis, 3rd ed., New York NY : John Wiley and Sons.
  33. Oral, M., Kettani, O., and Lang, P. (1991), A Methodology for Collective Evaluation and Selection of Industrial R&D Projects, Management Science, 37(7), 871-885. https://doi.org/10.1287/mnsc.37.7.871
  34. Paradi, J. C., Vela, S., and Yang, Z. (2004), Assessing Bank and Bank Branch Performance, In : Cooper, W. W., Seiford, L. M., and Zhu, J. ed., Handbook on Data Envelopment Analysis, Boston MA : Springer, 349-400.
  35. Park, S., Kim, H., Sul, W., Baek, D., and Khoe, K. (2011), Handbook on R&D Performance Evaluation, Korea : KSI.
  36. Parks, R. B. (1983), Technical Efficiency of Public Decision Making Units, Policy Studies Journal, 12(2), 337-346. https://doi.org/10.1111/j.1541-0072.1983.tb00275.x
  37. Pedraja-Chaparro, F., Salinas-Jimenez, J. and Smith, P. (1997), On the Role of Weight Restrictions in Data Envelopment Analysis, Journal of Productivity Analysis, 8(2), 215-230. https://doi.org/10.1023/A:1007715912664
  38. Rouse, P. and Putterill, M. (2003), An Integral Framework for Performance Measurement, Management Decision, 48(8), 791-805.
  39. Ruegg, R. and Feller, I. (2003), A Toolkit for Evaluating Public R&D Investment : Models, Methods and Findings from ATP's First Decade, Economic Assessment Office, Advanced Technology Program, National Institute of Standards and Technology (NIST), U. S. Department of Commerce, Gaithersburg MD.
  40. Seiford, L. M. and Thrall, R. M. (1990), Recent Development in DEA : The Mathematical Programming Approach to Frontier Analysis, Journal of Econometrics, 46(1-2), 7-38. https://doi.org/10.1016/0304-4076(90)90045-U
  41. Sharma, S. and Thomas, V. J. (2008), Inter-country R&D Efficiency Analysis : An Application of Data Envelopment Analysis, Scientometrics, 76(3), 483-501. https://doi.org/10.1007/s11192-007-1896-4
  42. Sherman, H. D. and Gold, F. (1985), Bank Branch Operating Efficiency : Evaluation with Data Envelopment Analysis, Journal of Banking and Finance, 9(2), 297-315. https://doi.org/10.1016/0378-4266(85)90025-1
  43. Simar, L. and Wilson, P. W. (2000), A General Methodology for Bootstrapping in Non-parametric Frontier Models, Journal of Applied Statistics, 27(6), 779-802. https://doi.org/10.1080/02664760050081951
  44. Thompson, R. G., Langemeier, L. N., Lee, C. T., Lee, E., and Thrall, R. M. (1990), The Role of Multiplier Bounds in Efficiency Analysis with Application to Kansas Farming, Journal of Econometrics, 46(1-2), 93-108. https://doi.org/10.1016/0304-4076(90)90049-Y
  45. W. K. Kellogg Foundation (WKKF) (2004), W. K. Kellogg Foundation Logic Development Guide, Battle Creek MI.
  46. Wholey, J. S. (1983), Evaluation and Effective Public Management, Boston MA : Little Brown.
  47. Wholey, J. S. (1987), Evaluability Assessment : Developing Program Theory, Special Issue : Using Program Theory in Evaluation, New Directions for Program Evaluation, 1987(33), 77-92.
  48. Wong, Y-H. B. and Beasley, E. (1990), Restricting Weight Flexibility in Data Envelopment Analysis, Journal of Operational Research Society, 41(9), 829-835. https://doi.org/10.1057/jors.1990.120
  49. Wu, W., Tsai, H., Cheng, K., and Lai, M. (2006), Assessment of Intellectual Capital Management in Taiwanese IC Design Companies : Using DEA and the Malmquist Productivity Index, R&D Management, 36(5), 531-545. https://doi.org/10.1111/j.1467-9310.2006.00452.x
  50. Zhu, J. (2003), Quantitative Models for Performance Evaluation and Benchmarking : Data Envelopment Analysis With Spreadsheets and DEA Excel Solver, Boston MA : Springer.