• Journal of Korea Technology Innovation Society
• /
• v.12 no.4
• /
• pp.819-839
• /
• 2009

Proposed increases to the government's R&D budget should be discussed based on merits of meeting efficiency and effectiveness criteria. The evaluation of the national R&D budget and related programs are performed in two areas: a system of R&D budget coordination and allocation, and a system of R&D program performance. This paper mainly focuses on the operational areas of R&D budget evaluation system with a focus on their impact to efficiency and effectiveness. The core view point for a R&D budget evaluation system involves two directions: Firstly, to detail the relationships between the later stage (ex. post) activities such as, program survey, analysis, and program performance evaluation, with the budget evaluation. Secondly, to critically oversee all R&D coordination procedures with a different perspective. Budgeting is generally known as a serial process of policy making, planning and executing. It is highly desirable for the budget to be allocated to, and spent by, specific programs as planned, and that each plan be aligned with a specific policy. As such, a strong relevance between the program structure and budget code system is integral to successful execution. It should be performed using a decision making system which closely examines the link between policy and budget. It is also recommended that systematic relationships be maintained among budget coordination and allocation, performance evaluations of policy and program levels, and program survey and analysis system, and that their operational schedule should be reviewed comprehensively as a one integrated system. The National Science and Technology Council is expected to play a major and practical role as the center of policy planning and should be supported by the objective and unbiased system which covers overall process from policy making to program evaluation. Finally, increased utilization of contents, timely program survey and analysis, and accurate of activity scheduling of budget coordination and allocation, and diligent program performance evaluation all contribute towards a more efficient and effective overall evaluation system.

• Journal of Korea Technology Innovation Society
• /
• v.18 no.4
• /
• pp.539-560
• /
• 2015

Korea has enacted the Performance Evaluation law on the R&D Programs since 2006. This study categorizes program size, program history, sustainability, self-evaluation result as antecedents, and analyze the relationship between the antecedents and Meta-evaluation result, the change rate of appropriations. The result shows that program size, program history, self-evaluation result have significant effect to the Meta-evaluation result. And Meta-evaluation result has significant effect to the change rate of appropriations. This study has implication that the performance evaluation system shows significant effect to the appropriations.

• Agribusiness and Information Management
• /
• v.3 no.1
• /
• pp.38-66
• /
• 2011

The dissemination process of agricultural research and development (R&D) results has somewhat different characteristics from that of typical R&D results. However, these characteristics are not adequately considered on the basis of an examination of the current performance system, the resulting management plans, and strategies for the application and dissemination of the results of agricultural R&D in Korea. The performance evaluation indicator exposed the problem of the inadequate consideration of the characteristics of each of these areas, particularly the lack of unified R&D-related institutions and the inadequacy of the system to monitor outcomes. To address these shortcomings in the agricultural R&D programs in Korea, the policies pertaining to agricultural R&D performance, results management, and dissemination in the U.S. and Japan were examined. Based on these investigations, we proposed strategies to improve the agricultural R&D policies in Korea.

• Journal of the Korean Regional Science Association
• /
• v.32 no.4
• /
• pp.3-18
• /
• 2016

National research and development(R&D) project plays an important role in the economic growth of a country as well as a company. This article aim to examine the development of the performance evaluation framework for assessing the contribution of national research and development(R&D) program to productivity growth. The main idea of this paper begins with the employment of between-group analysis for evaluating the returns to R&D. The method of this study is based on the comparison of productivity growth between different groups with time series data, not cross-sectional approach with one period. In these regards, backward effect and forward effect are operationally defined as a measurement for contributions of R&D in this study. The results of analysis gives that the outcomes of between-group analysis shows the lower performance of R&D compared to the within-group analysis. while the difference was not especially evident with the case of operating profit variable, but sales variable.

• STI Policy Review
• /
• v.4 no.2
• /
• pp.41-54
• /
• 2013

R&D is inherently an uncertain endeavor, yet now more than ever those performing R&D with public funding are called upon to clarify the utility of their research. Calls for public accountability are mounting with the increase in constraints on government budgets due to the recent worldwide economic recession, in response to which both policymakers and researchers pay much more attention to rigorously assessing publicly funded R&D. A key issue complicating R&D evaluation in these circumstances is how to adequately account for the nature and degree of risk involved in a given R&D program or project. This study deliberates on certain issues involving the measurement of ex-ante risk in public R&D evaluation: (i) information asymmetry between R&D sponsors and performers, (ii) ambiguity in the measurement of returns in both prospective and retrospective evaluation, and (iii) the dilemma between measurement error and omitted variable bias for empirical estimation of R&D performance. The study then presents an analysis of hypothetical evaluation results that apply risk-relevant weights to the annual evaluation outcomes of South Korea's national R&D programs funded during 2006~2012. In this counterfactual re-evaluation of public R&D program performance, high-risk R&D programs turn out to receive higher evaluation than non-high-risk programs. The current study suggests that R&D evaluation ignoring ex-ante risk is not only conceptually invalid since R&D activities are intrinsically uncertain endeavors, but unfair as R&D performers are asked to be accountable for the results that were in fact out of their reach.

• Journal of Internet Computing and Services
• /
• v.24 no.6
• /
• pp.31-39
• /
• 2023

The purpose of this study is to propose a scientific performance evaluation framework for measuring and managing the overall outcome of complex types of projects that are linked to public demand-based commercialization, such as information system projects and public procurement, in integrated national R&D projects. In the case of integrated national R&D projects that involve multiple research institutes to form a single final product, and in the case of demand-based demonstration and commercialization of the project results, the existing evaluation system that evaluates performance based on the short-term outputs of the detailed tasks comprising the R&D project has limitations in evaluating the mid- and long-term effects and practicality of the integrated research products. (Moreover, as the paradigm of national R&D projects is changing to a mission-oriented one that emphasizes efficiency, there is a need to change the performance evaluation of national R&D projects to focus on the effectiveness and practicality of the results.) In this study, we propose a performance evaluation framework from a structural perspective to evaluate the completeness of each national R&D project from a practical perspective, such as its effectiveness, beyond simple short-term output, by utilizing the Hourglass model. In particular, it presents an integrated performance evaluation framework that links the top-down and bottom-up approaches leading to Tool-System-Service-Effect according to the structure of R&D projects. By applying the proposed detailed evaluation indicators and performance evaluation frame to actual national R&D projects, the validity of the indicators and the effectiveness of the proposed performance evaluation frame were verified, and these results are expected to provide academic, policy, and industrial implications for the performance evaluation system of national R&D projects that emphasize efficiency in the future.

• Journal of Technology Innovation
• /
• v.30 no.3
• /
• pp.35-64
• /
• 2022

Currently, The budget expenditure for national R&D has been on the rise, and the quantitative performance in national R&D programs has been increasing accordingly. However, a growth of qualitative performance can not reach to an upward trend of quantitative performance. For a poor qualitative performance such as patent grades and technologies with low efficiency of technology transfer, the performance evaluation centered on "number of cases" is necessary to improve and start to introduce various "qualitative performance indicators". This paper focuses on "Triadic Patent Families" being measurable the overseas competitiveness of patent applied in major countries(the US, Europe and Japan). It analyzes the influencing factors to evaluate the quality of qualitative performance of R&D program. The result of this analysis shows a significant effect on the creating of triadic patent families within factors of R&D scale and project characteristics. This paper is significantly meaningful to expand insufficient triadic patent families to the target of interest rather than early study relatively, and to analyze the influencing factors of the creation of triadic patent families.

• Journal of Technology Innovation
• /
• v.5 no.2
• /
• pp.126-154
• /
• 1997

This paper addresses a methodology for monitoring the first phase('92-94') progress and its implication fo Highly Advanced National R&D Projects (HAN or G7 project) in Korea. The authors suggest a three-stage life cycle model for the management of national R&D program; planning stage, implementing & monitoring stage, and evaluating & harvesting stage. The interim evaluation, the main focuses of this paper, is one of the key issues for the effective management of national R&D program at the implementing & monitoring stage. The conceptual framework of the interim evaluation was developed both through reviewing related literature and considering the characteristics of the HAN program. A stepwise procedure of interim evaluation was introduced: 'initial screen' and 'detailed examination'. At initial screen all 11 projects were reviewed in terms of both their relevance and performance in order to decide whether further detailed scrutiny is needed or not. If there exist critical problems on the relevance or on the intermediate progress of each project, it is to be under further detailed examination. Evaluation was initiated by the G7 Planning and Evaluation Committee, which was established for advising, directing, and monitoring the HAN Projects, with the help of large number of experts from industries, universities, and research institutes. The monitoring results were reported to the related ministries and project coordinating agencies, and reflected in the second phase.

• Journal of Digital Convergence
• /
• v.11 no.4
• /
• pp.77-90
• /
• 2013

This study is to seek measures for streamlining the evaluation system by analyzing the examples of advanced countries. This study was conducted, based on the relevant government policy documents and previous research papers, in the manner of comparing and analyzing the issues and cases regarding the government evaluation system of public-funded R&D programs in selected countries. In Korea, a national R&D performance evaluation system was introduced by law in 2006. In United States, Obama government has enhanced government performance management by signed into law the GPRA Modernezation Act of 2010. The Japanese Government issued a newly revised Guideline on National R&D Evaluation in late 2012. European Union has also been constantly updating its evaluation system for Framework Programmes(FPs) begun since 1984. This study gets some insights from the recent development of R&D evaluation in the other countries. It is important for the government R&D evaluation system to be in line with national S&T policy and agency's mission. In micro views, specific evaluation approaches and methods by types of various R&D programs should be more illustratively developed. Additionally it is suggested to monitor recent trends and techniques on R&D evaluation by participating in activities to communicate and share their knowledge and experiences in international evaluation research networks.

• Journal of the Korean Regional Science Association
• /
• v.31 no.4
• /
• pp.71-90
• /
• 2015

This article aims to explore the performance evaluation framework for assessing the contribution of national research and development(R&D) program to productivity growth. The key idea starts with the development of methodology for evaluating the returns to R&D. The method of this study is based on the comparison of productivity growth between different periods, not cross-sectional approach with one period. In this context, backward effect and forward effect are operationally defined as a measurement for contributions of R&D in this paper. The analysis results gives that the backward effect of R&D is greater than the forward effect, while the difference was especially evident with the case of sales variable, not operating profit variable.