• 산업경영시스템학회지
• /
• 제32권3호
• /
• pp.59-70
• /
• 2009

This study is the result of meta evaluation for the self evaluation of defense R&D programs in Korea by using meta evaluating indicators. The overall meta evaluation result of defense R&D programs gained 74.3 points out of 100, so it was evaluated as 'Good'. But it demonstrated that further improvement for overall system of defense R&D programs evaluation is required. And especially, it demonstrated that more alternatives are necessary in order to improve the utilizations and the feedbacks of evaluation results. The evaluation context component gained 80.2 points out of 100, so it was evaluated as 'Very Good'. The evaluation input component gained 73.1 points out of 100, so it was evaluated as 'Good'. The evaluation process component gained 74.8 points out of 100, so it was evaluated as 'Good'. And the evaluation outcome component gained 69.0 points out of 100, so it was evaluated as 'Good'. Basic model of meta evaluation was derived from the literature review and brain storming. And this meta evaluation model was determined by adopting the result of experts who performed evaluations for defense R&D programs in recent years. The reliability of components and items was verified by Cronbach's a coefficient. It was over 0.6 in evaluation components and items. And the reliability of evaluation context was 0.877, that of evaluation input was 0.755, that of evaluation process was 0.755, that of evaluation output was 0.755 respectively. From the analysis, it is attempted to identify possible problems and to find out the ways of improvements related to the self evaluation system of defense R&D programs. The ultimate objective of this study is to manage the programs effectively and improve the reliability and the objectiveness of the defense R&D programs.

• 과학기술학연구
• /
• 제14권1호
• /
• pp.117-137
• /
• 2014

최근 출연(연)의 사회문제 해결형 R&D 사업이 점차 증가하고 있다. 그러나 출연(연)은 사회문제 해결에 적합한 R&D 시스템을 구축하지 못하여 많은 시행착오를 겪고 있다. 이에 본 연구에서는 사회문제 해결형 R&D 수행에 있어 주요 영향요인으로 작용하고 있는 출연(연)의 평가시스템 개선방향에 대해서 논의하고자 하였다. 이를 위해 먼저 사회문제 해결형 R&D에 관해 고찰하고, 출연(연)의 평가시스템 현황 및 사회문제 해결형 R&D 관점에서 한계를 도출한 후, 효과적 추진을 위한 개선방향을 제시하였다. 그리고 마지막으로 본 연구가 주는 함의와 한계를 논의함으로써 마무리하였다. 본 연구는 사회문제 해결 관점의 평가지표를 제시하고 활용방안을 모색한 최초의 연구로서 향후 관련분야 연구의 초석이 될 것으로 기대한다.

• 기술혁신학회지
• /
• 제13권1호
• /
• pp.44-67
• /
• 2010

국가 R&D사업은 적절한 예산 규모 산정이 어렵기 때문에 참고 가능한 해당 사업의 성과 및 평가정보가 적시에 지원될 경우 판단의 오류를 최소화 할 수 있을 것으로 기대할 수 있다. 그러나 현실적으로는 국가 R&D사업에 대한 성과평가 결과가 의도한 대로 충분히 활용되지 못하고 있는 실정인데, 그 이유로는 크게 두 가지 문제점을 들 수 있다. 첫째, 국가 R&D사업의 성과 및 평가 정보와 관련 예산정보가 적시에 연계 활용되지 못하고 있다. 둘째, 이를 지원하는 시스템간의 연계가 구현되어 있지 않다. 따라서 본 연구에서는 국가 R&D사업의 현황 분석을 신정부 출범 이전과 이후의 제도적, 시스템적 관점에서 고찰하였고 이를 통하여 국가 R&D성과평가와 예산 정보가 연계 가능하도록 개선 프로세스를 제안하고자 하였다.

• 디지털융복합연구
• /
• 제11권4호
• /
• pp.77-90
• /
• 2013

이 연구의 목적은 주요국의 정부 연구개발 평가제도 개선을 위한 최근 경향을 살펴 봄으로써 우리나라의 R&D평가제도 선진화를 위한 시사점을 모색하기 위함이다. 이 연구는 우리나라의 연구개발평가제도의 현황과 과제를 살펴보고, R&D 투자 상위국의 최근 연구개발사업 평가제도와 비교 분석하는 방식으로 수행되었다. 우리나라는 2006년 국가연구개발 성과평가법이 제정됨에 따라 연구개발사업 평가제도가 도입 운영되고 있다. 미국은 오바마 정부가 들어서면서 종전의 정부성과관리법을 개정('GPRA 현대화법 2010')하여 중점성과목표 관리제도를 강화하였으며, 일본은 2012년 말 '국가연구개발평가에 관한 대강적 지침'을 개정하여 연구개발 프로그램 및 결과지표 개념을 도입하였다. EU에서는 1984년부터 시작된 R&D 프레임워크프로그램(FP)에 대한 평가체계를 지속적으로 개선해 나가고 있다. 주요국의 연구개발 평가와 관련된 최근 동향으로부터 다음과 같은 시사점을 찾을 수 있다. 현행 연구개발사업 평가가 국가 과학기술정책과 기관의 고유임무 성과평가와도 연계성을 가질 수 있는 평가체계로 발전시켜 나가는 것이 중요하다. 또한 국제적인 평가연구 네트워크에 지속적인 참여 확대와 교류 활동을 통해 R&D 평가에 관한 선진 기법들을 적극적으로 수집하고, 나아가 R&D 성과관리에 관한 역량강화 프로그램과 체계적인 평가전문가 인력육성 교육시스템도 필요하다.

• Journal of Information Science Theory and Practice
• /
• 제10권spc호
• /
• pp.96-111
• /
• 2022

With the rapid development of the Fourth Industrial Revolution and digital transformation, scientific and technological innovation measures are being devised to overcome Korea's low-growth, high-cost structure. Accordingly, by examining the R&D investment evaluation system of R&D PIE (R&D Platform for Investment and Evaluation), which has been promoted by the Korean government in response to the Fourth Industrial Revolution, from the perspective of R&D transformation, this study aims to explore a new path for a sustainable national science and technology innovation system following digital transformation. In particular, from the perspective of R&D PIE, a MLP (Multi-level Perspective), which had been conducted as an abstract theoretical study, was attempted with specific cases and analysis for each of the three layers: niche, landscape, and regime. In conclusion, R&D PIE was intended to elevate the abstract R&D investment evaluation system to a platform that leads innovation in the digital space of the Fourth Industrial Revolution. In addition, it was confirmed that the R&D PIE could be replaced or enhanced as a platform for innovation in response to the Fourth Industrial Revolution, thereby providing an alternative to job creation and an escape from economic crisis.

• 기술혁신연구
• /
• 제3권1호
• /
• pp.55-83
• /
• 1995

This paper addresses three central decision-making problems frequently encountered in developing evaluation checklists for ex-post evaluation of R&D projects at private R&D centers: 1) what kind of evaluation elements and factors should be included in the evaluation forms\ulcorner 2) how much weight should be assigned to each evaluation factor and element\ulcorner and 3) could the same evaluation elements and factors with weights be applied all the project type(research, development, and engineering)\ulcorner To answer these questions, we identified the evaluation elements/factors based on the review of relevant literature and measured the perceptions of researchers, R&D managers, and top managements of the three private R&D centers to get the information about the importance levels of the elements and the weights of the factors for each of the three project types. The findings include that 1) all the evaluation elements derived from the literature exhibited high importance levels, and 2) the importance levels of the elements and the weights of the factors were quite varied along the project types. Theoretical and practical implications of the findings for ex-post evaluation of R&D projects are discussed.

• Asian Journal of Innovation and Policy
• /
• 제9권3호
• /
• pp.281-307
• /
• 2020

A variety of approaches are being applied to improve the existing ex-ante evaluation by expert panels in publicly funded R&D. While the objective evaluation criteria are constantly being improved to screen and select the superior projects, alternative approaches such as random prioritization and logical modeling are also underway to overcome the conservative bias of reviewers and to secure disruptive innovation. This study intends to find critical implications for ex-ante evaluation of public R&D system from the comparison of Indonesia and Korea. For the comparative analysis, literature review and expert in-depth interviews are conducted on the national R&D system and the selection evaluation process. In Korea, the selection criteria of projects are legally promulgated for establishing an objective evaluation system, and at the program level, the major considerations in the planning process are specified by Presidential Decree. On the other hand, while Indonesia conducts R&D in 47 strategic fields largely by public research institutes (PRI) based on the non-competitive government contributions. This study draws out implications of institutionalizing the planning process at the level of program, and of increasing the ratio of contract-based competitive funding at the level of project in the national R&D portfolio.

• 한국군사과학기술학회지
• /
• 제12권1호
• /
• pp.70-79
• /
• 2009

Currently, the investment on Research and Development(R&D) for defense core technology is continuously increasing to maintain advanced country's R&D level. In accordance with this trend, the national defense R&D investment management system is being more strengthening, and importance of performance evaluation is being emphasized more and more. However, the existing indicators of performance evaluation for defense core technology R&D is limited in efficient management because of the qualitative evaluation items and subjective allocation of points. In this paper, we developed a novel indicator for defense core technology R&D performance evaluation through domestic and foreign related literature and inquiry, brainstorming, and analysis techniques by Stepwise Multiple linear Regression(SMR) and Analytic Hierarchy Process(AHP). And we verified effectiveness of proposed indicators of performance evaluation by comparing with the existing evaluation indicators. Our proposed indicators for performance evaluation will create superiority performance on defense R&D fields.

• 산업경영시스템학회지
• /
• 제32권2호
• /
• pp.29-37
• /
• 2009

National R&D programs play an important role in the development of a country in this age of the knowledge economy. Since many numbers of R&D programs compete for limited resources such as national R&D budget, the R&D program evaluation problem is a challenging decision-making problem faced by decision makers that deal with R&D management. In this sense, DEA(Data Envelopment Analysis) has been regarded as one of the most widely accepted methods to measure the relative efficiency of productivity of R&D programs. DEA is a methodology to measure and to evaluate the relative efficiency of a homogeneous set of decision-making units(DMUs) in a process which uses multiple inputs to produce multiple outputs. However, the sample of the R&D programs could consist of two or more naturally occurring subsets, thus exhibiting clear signs of heterogeneity such as different objectives. In such situations, the fairness of DEA is limited, for the nature of the relative efficiency of a DMU is likely to be influenced by its membership in a particular subset of the sample. In this study, we propose a methodology AI-DEA(attainment index DEA) allowing for reflecting decision maker's subjective judgement on difference among different subsets of R&D programs which have heterogeneous objectives. This methodology combines AHP and Delphi in order to decide the attainmnet index of each DMU for each outputs, and apply them to DEA model. We illustrate the proposed approach with a pilot evaluation of 13 programs involving 6 different subsets of Korean National R&D programs and compares the results of the original DEA model and AI-DEA model.

• 기술혁신학회지
• /
• 제14권4호
• /
• pp.937-964
• /
• 2011

이 연구는 농업 연구개발사업의 객관적인 성과와는 달리 농촌진흥청의 역할에 대한 논의가 끊이지 않고 있는 문제의식에서 출발하고 있다. 이에 따라 농업 연구개발사업의 핵심 주체인 농촌진흥청의 내부 연구개발 평가체계를 대상으로 평가기조, 평가자원, 평가수행, 평가활용이라는 4가지 관점에서 메타 평가를 수행하였다. 연구결과는 평가기조에 해당하는 평가대상의 설정이 최하위 수준인 세부 연구과제로 설정되어 있다는 점이 가장 큰 문제로 분석되었다. 따라서 이 연구는 우선적으로 평가목적과 평가대상 등의 평가기조를 과학기술정책 및 평가정책 등의 외부의 정책환경 변화와 조응할 수 있는 방향으로 전환함으로서 현행 농업 연구개발사업의 평가제도를 성과지향적으로 발전시키기 위한 정책대안을 논의하였다.