Browse > Article
http://dx.doi.org/10.7232/JKIIE.2014.40.1.084

Identification of DEA Determinant Input-Output Variables : an Illustration for Evaluating the Efficiency of Government-Sponsored R&D Projects  

Park, Sungmin (Department of Business Administration, Baekseok University)
Publication Information
Journal of Korean Institute of Industrial Engineers / v.40, no.1, 2014 , pp. 84-99 More about this Journal
Abstract
In this study, determinant input-output variables are identified for calculating Data Envelopment Analysis (DEA) efficiency scores relating to evaluating the efficiency of government-sponsored research and development (R&D) projects. In particular, this study proposes a systematic framework of design and analysis of experiments, called "all possible DEAs", for pinpointing DEA determinant input-output variables. In addition to correlation analyses, two modified measures of time series analysis are developed in order to check the similarities between a DEA complete data structure (CDS) versus the rest of incomplete data structures (IDSs). In this empirical analysis, a few DEA determinant input-output variables are found to be associated with a typical public R&D performance evaluation logic model, especially oriented to a mid- and long-term performance perspective. Among four variables, only two determinants are identified : "R&D manpower" ($x_2$) and "Sales revenue" ($y_1$). However, it should be pointed out that the input variable "R&D funds" ($x_1$) is insignificant for calculating DEA efficiency score even if it is a critical input for measuring efficiency of a government-sonsored R&D project from a practical point of view a priori. In this context, if practitioners' top priority is to see the efficiency between "R&D funds" ($x_1$) and "Sales revenue" ($y_1$), the DEA efficiency score cannot properly meet their expectations. Therefore, meticulous attention is required when using the DEA application for public R&D performance evaluation, considering that discrepancies can occur between practitioners' expectations and DEA efficiency scores.
Keywords
Data Envelopment Analysis; Design of Experiments and Analysis; Determinant Input-output Variables; Efficiency; R&D Performance Evaluation;
Citations & Related Records
연도 인용수 순위
  • Reference
1 Paradi, J. C., Vela, S., and Yang, Z. (2004), Assessing Bank and Bank Branch Performance, In : Cooper, W. W., Seiford, L. M., and Zhu, J. ed., Handbook on Data Envelopment Analysis, Boston MA : Springer, 349-400.
2 Park, S., Kim, H., Sul, W., Baek, D., and Khoe, K. (2011), Handbook on R&D Performance Evaluation, Korea : KSI.
3 Parks, R. B. (1983), Technical Efficiency of Public Decision Making Units, Policy Studies Journal, 12(2), 337-346.   DOI   ScienceOn
4 Pedraja-Chaparro, F., Salinas-Jimenez, J. and Smith, P. (1997), On the Role of Weight Restrictions in Data Envelopment Analysis, Journal of Productivity Analysis, 8(2), 215-230.   DOI
5 Rouse, P. and Putterill, M. (2003), An Integral Framework for Performance Measurement, Management Decision, 48(8), 791-805.
6 Ruegg, R. and Feller, I. (2003), A Toolkit for Evaluating Public R&D Investment : Models, Methods and Findings from ATP's First Decade, Economic Assessment Office, Advanced Technology Program, National Institute of Standards and Technology (NIST), U. S. Department of Commerce, Gaithersburg MD.
7 Seiford, L. M. and Thrall, R. M. (1990), Recent Development in DEA : The Mathematical Programming Approach to Frontier Analysis, Journal of Econometrics, 46(1-2), 7-38.   DOI
8 Sharma, S. and Thomas, V. J. (2008), Inter-country R&D Efficiency Analysis : An Application of Data Envelopment Analysis, Scientometrics, 76(3), 483-501.   DOI
9 Sherman, H. D. and Gold, F. (1985), Bank Branch Operating Efficiency : Evaluation with Data Envelopment Analysis, Journal of Banking and Finance, 9(2), 297-315.   DOI   ScienceOn
10 Simar, L. and Wilson, P. W. (2000), A General Methodology for Bootstrapping in Non-parametric Frontier Models, Journal of Applied Statistics, 27(6), 779-802.   DOI   ScienceOn
11 Thompson, R. G., Langemeier, L. N., Lee, C. T., Lee, E., and Thrall, R. M. (1990), The Role of Multiplier Bounds in Efficiency Analysis with Application to Kansas Farming, Journal of Econometrics, 46(1-2), 93-108.   DOI   ScienceOn
12 W. K. Kellogg Foundation (WKKF) (2004), W. K. Kellogg Foundation Logic Development Guide, Battle Creek MI.
13 Wholey, J. S. (1983), Evaluation and Effective Public Management, Boston MA : Little Brown.
14 Wholey, J. S. (1987), Evaluability Assessment : Developing Program Theory, Special Issue : Using Program Theory in Evaluation, New Directions for Program Evaluation, 1987(33), 77-92.
15 Farris, J. A., Groesbeck, R. L., Aken, E. M. V., and Letens, G. (2006), Evaluating the Relative Performance of Engineering Design Projects : A Case Study Using Data Envelopment Analysis, IEEE Transactions on Engineering Management, 55(3), 471-482.
16 Wong, Y-H. B. and Beasley, E. (1990), Restricting Weight Flexibility in Data Envelopment Analysis, Journal of Operational Research Society, 41(9), 829-835.   DOI
17 Wu, W., Tsai, H., Cheng, K., and Lai, M. (2006), Assessment of Intellectual Capital Management in Taiwanese IC Design Companies : Using DEA and the Malmquist Productivity Index, R&D Management, 36(5), 531-545.   DOI   ScienceOn
18 Zhu, J. (2003), Quantitative Models for Performance Evaluation and Benchmarking : Data Envelopment Analysis With Spreadsheets and DEA Excel Solver, Boston MA : Springer.
19 Government Performance Results Act (GPRA) (1993), U.S. Office of Management and Budget, Available online at : http://www.whitehouse.gov/omb/mgmt-gpra/gplaw2m(accessed 1 October 2012).
20 Guan, J. and Chen, K. (2010), Modeling Macro-R&D Production Frontier Performance : An Application to Chinese Province-level R&D, Scientometrics, 82(1), 165-173.   DOI
21 Hsu, F. M. and Hsueh, C. C. (2009), Measuring Relative Efficiency of Government-sponsored R&D Projects : A Three-stage Approach, Evaluation and Program Planning, 32(2), 178-186.   DOI   ScienceOn
22 Lee, H., Park, Y. and Choi, H. (2009), Comparative Evaluation of Performance of National R&D Programs with Heterogeneous Objectives : A DEA Approach, European Journal of Operational Research, 196(3), 847-855.   DOI   ScienceOn
23 Ministry of Science and Technology (MST)․Korea Institute of Science and Technology Evaluation and Planning (KISTEP) (2007), A Study of the Methodology for Follow-up Evaluation of National R&D Programs, Korea : MST․KISTEP.
24 McLaughlin, J. A. and Jordan, G. B. (1999), Logic Models : A Tool for Telling Your Program's Performance Story, Evaluation and Program Planning, 22(1), 65-72.   DOI   ScienceOn
25 Meng, W., Hu, Z. H., and Liu, W. B. (2006), Efficiency Evaluation of Basic Research in China, Scientometrics, 69(1), 85-101.   DOI   ScienceOn
26 Ministry of Knowledge and Economy (MKE)․National IT Industry Promotion Agency (NIPA) (2012), Performance Analysis on Information and Communication Promotion Fund (VII), Korea : MKE․NIPA.
27 MinitabR (2005), MinitabR Release 14.20 StatGuide, State College PA : Minitab Inc.
28 Montgomery, D. C., Peck, E. A., and Vining, G. G. (2001), Introduction to Linear Regression Analysis, 3rd ed., New York NY : John Wiley and Sons.
29 Oral, M., Kettani, O., and Lang, P. (1991), A Methodology for Collective Evaluation and Selection of Industrial R&D Projects, Management Science, 37(7), 871-885.   DOI   ScienceOn
30 Bitman, W. R. and Sharif, N. (2008), A Conceptual Framework for Ranking R&D Projects, IEEE Transactions on Engineering Management, 55(2), 267-278.
31 Callen, J. L. (1991), Data Envelopment Analysis : Partial Survey and Applications for Management Accounting, Journal of Management Accounting Research, 3, 35-56.
32 Charnes, A., Cooper, W. W. and Rhodes, E. (1978), Measuring the Efficiency of Decision Making Units, European Journal of Operational Research, 2(6), 429-444.   DOI   ScienceOn
33 Charnes, A., Clark, C. T., Cooper, W. W., and Golany, B. (1985), A Developmental Study of Data Envelopment Analysis in Measuring the Efficiency of Maintenance Units in the US Air Forces, In Thompson, R. G. and Thrall, R. M. (Ed.), Annals of Operations Research, 2(1), 95-112.
34 Charnes, A. and Cooper, W. W. (1980), Auditing and Accounting for Program Efficiency and Management Efficiency in Not-for-profit Entities, Accounting, Organizations and Society, 5(1), 87-107.   DOI   ScienceOn
35 Charnes, A., Cooper, W. W., Huang, Z. M. and Sun, D. B. (1990), Polyhedral Cone-ratio DEA Models with an Illustrative Application to Large Commercial Banks, Journal of Econometrics, 46(1/2), 73-91.   DOI   ScienceOn
36 Charnes, A., Cooper, W. W., and Rhodes, E. (1981), Evaluating Program and Managerial Efficiency : An Application of Data Envelopment Analysis to Program Follow Through, Management Science, 27(6), 668-697.   DOI   ScienceOn
37 Chiesa, V. and Masella, C. (1996), Searching for an Effective Measure of R&D Performance, Management Decision, 34(7), 49-57.
38 Cooper, W. W., Seiford, L. M., and Tone, K. (2007), Data Envelopment Analysis : A Comprehensive Text With Models, Applications, References and DEA-Solver Software, 2nd ed., New York NY : Springer.
39 Cooper, W. W., Seiford, L. M., and Zhu, J. (2004), Handbook on Data Envelopment Analysis, Boston MA : Springer.
40 DEA-Solver-Pro (2012), Professional Version 8.0 with Excel 2010 on 64 bit Windows 7, Holmdel NJ : SAITECH, Inc.
41 Asmild, M., Paradib, J. C., Reesec, D. N., and Tamb, F. (2007), Measuring Overall Efficiency and Effectiveness Using DEA, European Journal of Operational Research, 178 (1), 305-321.   DOI   ScienceOn
42 Bickman, L. (1987), The Functions of Program Theory, Special Issue : Using Program Theory in Evaluation, New Directions for Program Evaluation, 1987(33), 5-18.
43 Abramo, G., D'Angelo, C. A., and Pugini, F. (2008), The Measurement of Italian Universities' Research Productivity by a Non Parametric-bibliometric Methodology, Scientometrics, 76(2), 225-244.   DOI
44 Allen, R., Athanassopoulos, A., Dyson, R. G., and Thanassoulis, E. (1997), Weights Restrictions and Value Judgements in Data Envelopment Analysis, Annals of Operations Research, 73(1), 13-34.   DOI
45 Banker, R. D., Charnes, A. and Cooper, W. W. (1984), Some Models for Estimating Technical and Scale Inefficiencies in Data Envelopment Analysis, Management Science, 30(9), 1078-1092.   DOI   ScienceOn
46 Banker, R. D., Conrad, R. F., and Strauss, R. P. (1986), A Comparative Application of Data Envelopment Analysis and Translog Methods : An Illustrative Study of Hospital Production, Management Science, 32(1), 30-44.   DOI   ScienceOn
47 Banker, R. D. and Morey, R. (1986a), Efficiency Analysis for Exogenously Fixed Inputs and Outputs, Operations Research, 34(4), 513-521.   DOI   ScienceOn
48 Banker, R. D. and Morey, R. (1986b), The Use of Categorical Variables in Data Envelopment Analysis, Management Science, 32(12), 1613-1627.   DOI   ScienceOn
49 Berenson, M. L., Levine, D. M., and Krehbiel, T. C. (2012), Basic Business Statistics, 12th ed., Boston MA : Pearson Education Limited.
50 Bessent, A., Bessent, W., Kennington, J., and Reagan, B. (1982), An Application of Mathematical Programming to Assess Productivity in the Houston Independent School District, Management Science, 28(12), 1355-1367.   DOI   ScienceOn