Journal of Korean Institute of Industrial Engineers (대한산업공학회지)

Korean Institute of Industrial Engineers (대한산업공학회)
권호 표지
DOI QR코드

DOI QR Code

Performance Evaluation of R&D Commercialization : A DEA-Based Three-Stage Model of R&BD Performance

연구개발 사업화 성과 평가 : DEA 기반 3단계 R&BD 성과 모형

  • Jeon, Ikjin (The Graduate School of Public Policy and Information Technology, Seoul National University of Science and Technology) ;
  • Lee, Hakyeon (Department of Industrial and Systems Engineering, Seoul National University of Science and Technology)
  • 전익진 (서울과학기술대학교 IT정책전문대학원) ;
  • 이학연 (서울과학기술대학교 글로벌융합산업공학과)
  • Received : 2015.03.05
  • Accepted : 2015.07.15
  • Published : 2015.10.15
Download PDF
⟨ Previous Next ⟩

Abstract

This study proposes a three-stage model of R&BD performance which captures commercialization outcomes as well as conventional R&D performance. The model is composed of three factors : inputs (R&D budgets and researchers), outputs (patents and papers), and outcomes (technical fees, products sales, and cost savings). Three stages are defined for each transformation process between the three factors : efficiency stage from input to output (stage 1), effectiveness stage from output to outcome (stage 2), and productivity stage from input to outcome (stage 3). The performance of each stage is measured by data envelopment analysis (DEA). DEA is a non-parametric efficiency measurement technique that has widely been used in R&D performance measurement. We measure the performance of 171 projects of 6 public R&BD programs managed by Seoul Business Agency using the proposed three-stage model. In order to provide a balanced and holistic view of R&BD performance, the R&BD performance map is also constructed based on performance of efficiency and productivity stages.

Keywords

References

  1. Banker, R. D., Charnes, A., and Cooper, W. W. (1984), Some models for estimating technical and scale inefficiency in data envelopment analysis, Management Science, 30(9), 1078-1092. https://doi.org/10.1287/mnsc.30.9.1078
  2. Banker, R. D., Charnes, A., Cooper, W. W., Swarts, J., and Thomas, D. (1989), An introduction to data envelopment analysis with some of its models and their uses, Research in Governmental and Nonprofit Accounting, 5, 125-163.
  3. Bickman, L. (1987), The functions of program theory, New Directions for Program Evaluation, Jossey-Bass, San Francisco, 33, 5-18.
  4. Bonaccorsi, A. and Daraio, C. (2003), A robust nonparametric approach to the analysis of scientific productivity, Research Evaluation, 12(1), 47-69. https://doi.org/10.3152/147154403781776726
  5. Boussofiane, A., Dyson, R. G., and Thanassoulis, E. (1991), Applied data envelopment analysis, European Journal of Operational Research, 52(1), 1-15. https://doi.org/10.1016/0377-2217(91)90331-O
  6. Charnes, A., Cooper, W. W., and Rhodes, E. (1978), Measuring efficiency of decision making units, European Journal of Operational Research, 2(6), 429-444. https://doi.org/10.1016/0377-2217(78)90138-8
  7. Chun, H. and Lee, H. (2013), A DEA-Based Portfolio Model for Performance Management of Online Games, Journal of the Korean Institute of Industrial Engineers, 39(4), 260-270. https://doi.org/10.7232/JKIIE.2013.39.4.260
  8. Chun, H. and Lee, H. (2014), Measuring Operational Efficiency of Korean Online Game Companies with DEA Window Analysis, Journal of the Korean Operations Research and Management Science Society, 39(3), 23-40. https://doi.org/10.7737/JKORMS.2014.39.3.023
  9. Cooper, W. W., Seiford, L. M., and Tone, K. (2007), Data envelopment analysis : A comprehensive text with models, applications, references and DEA-Solver Software, Second editions, 490, Springer.
  10. Eilat, H., Golany, B., and Shtub, A. (2006), Constructing and evaluating balanced portfolios of R&D projects with interactions : A DEA based methodology, European Journal of Operational Research, 172(3), 1018-1039. https://doi.org/10.1016/j.ejor.2004.12.001
  11. Farris, J. A., Groesbeck, R. L., Van-Aken, E. M., and Letens, G. (2006), Evaluating the relative performance of engineering design projects : A case study using data envelopment analysis, IEEE Transactions on Engineering Management, 53(3), 471-482. https://doi.org/10.1109/TEM.2006.878100
  12. Garg, K. C., Gupta, B. M., Jamal, T., Roy, S., and Kumar, S. (2005), Assessment of impact of AICTE funding on R&D and educational development, Scientometrics, 65(2), 151-160. https://doi.org/10.1007/s11192-005-0264-5
  13. Georghiou, L. (1999), Socio-economic effects of collaborative R&D-European experiences, Journal of Technology Transfer, 24(1), 69-79. https://doi.org/10.1023/A:1007724804288
  14. Guan, J. and Chen, K. (2012), Modeling the relative efficiency of national innovation systems, Research Policy, 41(1), 102-115. https://doi.org/10.1016/j.respol.2011.07.001
  15. Guan, J. and Wang, J. (2004), Evaluation and interpretation of knowledge production efficiency, Scientometrics, 59(1), 131-155. https://doi.org/10.1023/B:SCIE.0000013303.25298.ae
  16. Hsu, F. M. and Hsueh, C. C. (2009), Measuring relative efficiency of government-sponsored R&D projects : A three-stage approach, Evaluation and Program Planning, 32(2), 178-186. https://doi.org/10.1016/j.evalprogplan.2008.10.005
  17. Jeon, J., Kim, C., and Lee, H. (2011), Measuring efficiency of total productive maintenance(TPM) : a three-stage data envelopment analysis( DEA) approach, Total Quality Management and Business Excellence, 22(8), 911-924. https://doi.org/10.1080/14783363.2011.593865
  18. Keh, H. T. and Chu, S. (2003), Retail productivity and scale economies at the firm level : A DEA approach, Omega, 31(2), 75-82. https://doi.org/10.1016/S0305-0483(02)00097-X
  19. Keh, H. T., Chu, S., and Xu, J. (2006), Efficiency, effectiveness and productivity of marketing in services, European Journal of Operational Research, 170(1), 265-276. https://doi.org/10.1016/j.ejor.2004.04.050
  20. Kerssens-van Drongelen, I., Nixon, B., and Pearson, A. (2000), Performance measurement in industrial R&D, International Journal of Management Reviews, 2(2), 111-143. https://doi.org/10.1111/1468-2370.00034
  21. Kim, Y.-H. and Lim, H.-J. (2013), A Study on the Creative Economy and Diffusion of R&D, Korea Productivity Association, 27(2), 285-307.
  22. Kocher, M. G., Luptacik, M., and Sutter, M. (2006), Measuring productivity of research in economics : A cross-country study using DEA, Socio-Economic Planning Sciences, 40(4), 314-332. https://doi.org/10.1016/j.seps.2005.04.001
  23. Lee, C. and Cho, K. (2014), Efficiency Analysis and Strategic Portfolio Model of National Health Technology R&D Program Using DEA : Focused on Translational Research, Journal of the Korean Institute of Industrial Engineers, 40(2), 172-183. https://doi.org/10.7232/JKIIE.2014.40.2.172
  24. Lee, D., Bae, S., and Kang, J. (2006), Development of R&D Project Selection Model and Web-based R&D Project Selection System using Hybrid DEA/AHP Model, Journal of the Korean Institute of Industrial Engineers, 32(1), 18-28.
  25. Lee, H. and Park, Y. (2005), An international comparison of R&D efficiency : DEA approach, Asian Journal of Technology Innovation, 13(2), 207-221. https://doi.org/10.1080/19761597.2005.9668614
  26. Lee, H. and Shin, J. (2014), Measuring journal performance for multidisciplinary research : An efficiency perspective, Journal of Informetrics, 8(1), 77-88. https://doi.org/10.1016/j.joi.2013.10.004
  27. Lee, H., Park, Y., and Choi, H. (2009), Comparative evaluation of performance of national R&D programs with heterogeneous objectives : A DEA approach, European Journal of Operational Research, 196(3), 847-855. https://doi.org/10.1016/j.ejor.2008.06.016
  28. Linton, J. D., Morabito, J., and Yeomans, J. S. (2007), An extension to a DEA support system used for assessing R&D projects, R&D Management, 37(1), 29-36.
  29. Linton, J. D., Walsh, S. T., and Morabito, J. (2002), Analysis, ranking and selection of R&D projects in a portfolio, R&D Management, 32(2), 139-148. https://doi.org/10.1111/1467-9310.00246
  30. Liu, J. S. and Lu, W. M. (2010), DEA and ranking with the network-based approach : a case of R&D performance, Omega, 38(6), 453-464. https://doi.org/10.1016/j.omega.2009.12.002
  31. Meng, W., Hu, Z., and Liu, W. (2006), Efficiency evaluation of basic research in China, Scientometrics, 69(1), 85-101. https://doi.org/10.1007/s11192-006-0140-y
  32. Meng, W., Zhang, D., Qi, L., and Liu, W. (2008), Two-level DEA approaches in research evaluation, Omega, 36(6), 950-957. https://doi.org/10.1016/j.omega.2007.12.005
  33. Park, S. (2014), Identification of DEA Determinant Input-Output Variables, Journal of the Korean Institute of Industrial Engineers, 40(1), 84-99. https://doi.org/10.7232/JKIIE.2014.40.1.084
  34. Revilla, E., Sarkis, J., and Modrego, A. (2003), Evaluating performance of public-private research collaborations : A DEA analysis, Journal of the Operational Research Society, 54(2), 165-174. https://doi.org/10.1057/palgrave.jors.2601524
  35. Rousseau, S. and Rousseau, R. (1997), Data envelopment analysis as a tool for constructing scientometric indicators, Scientometrics, 40(1), 45-56. https://doi.org/10.1007/BF02459261
  36. Ruegg, R. and Feller, I. (2003), A toolkit for evaluating public R&D investment models, methods, and findings from ATP's first decade, National Institute of Standards and Technology, Technology Administration, 3-857, US Department of Commerce, Gaithersburg.
  37. Sharma, S. and Thomas, V. J. (2008), Inter-country R&D efficiency analysis : An application of data envelopment analysis, Scientometrics, 76(3), 483-501. https://doi.org/10.1007/s11192-007-1896-4
  38. Thompson, R. G., Langemeier, L. N., Lee, C. T., Lee, E., and Thrall, R. M.(1990), The role of multiplier bounds in efficiency analysis with application to Kansas farming, Journal of Econometrics, 46(1), 93-108. https://doi.org/10.1016/0304-4076(90)90049-Y
  39. Wang, E. C. and Huang, W. (2007), Relative efficiency of R&D activities : A cross-country study accounting for environmental factors in the DEA approach, Research Policy, 36(2), 260-273. https://doi.org/10.1016/j.respol.2006.11.004
  40. Zhang, A., Zhang, Y., and Zhao, R. (2003), A study of the R&D efficiency and productivity of Chinese firms, Journal of Comparative Economics, 31(3), 444-464. https://doi.org/10.1016/S0147-5967(03)00055-6

Cited by

  1. The Framework for the Strategy of Research & Business Development vol.44, pp.4, 2016, https://doi.org/10.7469/JKSQM.2016.44.4.785