• Journal of Korean Society of Industrial and Systems Engineering
• /
• v.35 no.2
• /
• pp.1-8
• /
• 2012

The primary goal of this study is to develop proper performance indicators for evaluating the results of industrial R&D program. We define R&D results include output, transfer, outcome, and impact in the performance of R&D program. Performance indicators are focused on the results of R&D commercialization, building the industrial foundations as performance proxy of R&D program of Ministry of Knowledge Economy. And we first try to design the coupled structure as logical model between R&D program goals and performance indicators. The coupled structure is represented with objectives of group unit of R&D program, implementation process, R&D practical stage, and the ultimate goal of R&D program.

• Knowledge Management Research
• /
• v.10 no.2
• /
• pp.65-84
• /
• 2009

The purpose of this study is to develop Meta Evaluation Indicators for Defense R&D Programs in Korea. At first, the four components of this meta evaluation model were designed, which are evaluation context, evaluation input, evaluation process and evaluation outcome. And fifty two indicators for this meta evaluation were developed by experts who performed evaluations for Defense R&D Programs. The reliability of components and items was verified by Cronbach's ${\alpha}$ coefficient. It was over 0.6 in all areas. The validity of components and items was verified by Factor Analysis. Analytic Hierarchy Process method was used in assigning the evaluation weight. The survey of twenty two evaluators participated in Defense R&D Programs showed that the Consistency Ratio was under 0.1 in evaluation components and items. In this study, an objective and reasonable set of Defense R&D Meta Evaluation Indicators was developed to increase the responsibility of Defense R&D Programs and improve the quality of evaluation results.

• IE interfaces
• /
• v.23 no.1
• /
• pp.78-88
• /
• 2010

In this paper we develop a model for the performance evaluation of defense R&D test development projects based on analytic hierarchy process. First, evaluation indicators are collected through the related literature survey and a delphi inquiry method. Second, stepwise multiple linear regression is used for developing a hierarchical structure for analytic hierarchy process in the evaluation model, which can make the selected evaluation indicators of the hierarchical structure independent. Also we verify the effectiveness of proposed indicators of the performance evaluation by comparing with the existing evaluation indicators. The developed indicators for the performance evaluation is more reasonable and practical than the previous indicators on defense R&D test development projects.

• Journal of Korea Technology Innovation Society
• /
• v.11 no.3
• /
• pp.376-399
• /
• 2008

Science and technology (S&T) is one of the most important elements in a nation's competitiveness. In an effort to strengthen their national competitiveness, all countries are focusing on upgrading the level of eir S&T. With these factors in mind, Korea has increased its support of national research and development (R&D). In recent years, this added support has resulted in an increased interest in the effectiveness of R&D. We have made continuous efforts to enhance the accountability and effectiveness of R&D by strengthening performance evaluation and considering R&D evaluation results during the budget review (appropriation) process. In order to change to a performance based system, we need to develop objective and scientific indicators to measure and evaluate the quality of the research performance of R&D programs. One of the primary research outcomes is publications. The impact factor of publications is widely used to evaluate overall journal quality and the quality of the papers published therein. However, the use of impact factors has been criticised because they can vary greatly when works from different subject areas are compared. In order to overcome this limitation, we have developed three kinds of qualitative indicators, which are functions of the impact factor. Two of these qualitative indicators, Modified Rank Normalized Impact Factor and Ordinal Rank Normalized Impact Factor, are based on order statistics (rank) for all journals from a specific specialty. The third qualitative indicator, Relative Field Impact Factor, uses the average impact factor of all journals within a subject category. We also suggest a quantitative indicator, Percentage of Contribution. In this study, we suggest 4 indicators and use them to evaluate the performance of outcomes from three R&D programs supported by the Ministry of Education, Science & Technology. We also perform a simulation study to verify the effectiveness of the proposed indicators. It can be shown that the proposed Ordinal Rank Normalized Impact Factor is the most reliable and effective indicator for comparing research performance across subject categories. However, we recommend using previous indicators in combination with the proposed indicators in this study for the research evaluation of R&D programs.

• IE interfaces
• /
• v.21 no.1
• /
• pp.96-108
• /
• 2008

This paper proposes the systematic design approach for developing the evaluation indicators that evaluate the candidate proposals in the national defense core-technology R&D projects. To improve the validity and fairness of the evaluation indicators, the existing evaluation process in a military and a public sector are surveyed and also the existing evaluation system of the core-technology R&D programs for the national defense is analysed and discussed. A new system for the evaluation indicators is designed by using the axiomatic design, factor analysis and the analytic hierarchy process. It is expected that the proposed evaluation indicators contribute to enhance the fairness and the reliability of the evaluation process for the proposal of the national defense core-technology R&D projects.

• Journal of the Korea Institute of Military Science and Technology
• /
• v.12 no.1
• /
• pp.70-79
• /
• 2009

Currently, the investment on Research and Development(R&D) for defense core technology is continuously increasing to maintain advanced country's R&D level. In accordance with this trend, the national defense R&D investment management system is being more strengthening, and importance of performance evaluation is being emphasized more and more. However, the existing indicators of performance evaluation for defense core technology R&D is limited in efficient management because of the qualitative evaluation items and subjective allocation of points. In this paper, we developed a novel indicator for defense core technology R&D performance evaluation through domestic and foreign related literature and inquiry, brainstorming, and analysis techniques by Stepwise Multiple linear Regression(SMR) and Analytic Hierarchy Process(AHP). And we verified effectiveness of proposed indicators of performance evaluation by comparing with the existing evaluation indicators. Our proposed indicators for performance evaluation will create superiority performance on defense R&D fields.

• International Journal of Quality Innovation
• /
• v.8 no.3
• /
• pp.156-172
• /
• 2007

The high-tech industry is the economic lifeline for Taiwan. Its characteristics are short product life cycle, rapid changes in the market, and a high obsolescence rate for new products. Under globalization, the high-tech industry has adopted Information Technology (IT) to shorten the manufacturing process, reduce costs and conduct product research and development (R&D) to increase the core competence of enterprises and achieve the goal of sustainable operations. Enterprises should actively strengthen their integration with internal and external resources and lead in R&D management to increase industrial operating performance. Effectively managing operations and R&D management evaluation in Taiwan's High-tech Industry has become a critical subject. This study adopted 4 major Balanced Scorecard (BSC) perspectives to establish the Total Performance Evaluation Indicators for the R&D management department in Taiwan's High-tech Industry. The Analytic Network Process (ANP) was applied to evaluate the overall performance of the R&D management department. The research framework is divided into 2 phases. The first phase is combined with the 4 major perspectives, Financial, Customer, Internal Business Process and Learning and Growth, as the related indicators for each measurement perspective. The Key Performance Indicators (KPI) were selected using Factor Analysis to identify the key factor from the complicated indicators. The relationship between the characteristics of each BSC's evaluation perspective is dependence and feedback. This study applied ANP to conduct the calculation and adjustment of correlation between each KPI, and determine on their relative weights for the objective KPI. The "Financial Perspective" for R&D management department in Taiwan's High-tech Industry focused on the budget achievement rate of R&D management. The weight indicator value is (0.05863). The "Customer Perspective" focused on problem-solving satisfaction. The weight value of this indicator is (0.17549). The "Internal Business Process Perspective" focused on the quantity and quality of R&D. The weight value of this indicator is (0.13506). The "Learning and Growth Perspective" focused on improving competence in the research personnel's professional techniques. The weight value of this indicator is (0.02789). From the total weighting indicators, the order of the Performance Indicators for the R&D management department in Taiwan's High-tech Industry is: (1) Customer Perspective; (2) Internal Business Process Perspective; (3) Financial Perspective; and (4) Learning and Growth Perspective.

• Journal of Korea Technology Innovation Society
• /
• v.15 no.4
• /
• pp.815-836
• /
• 2012

Corporate Laboratories have been a key driver of Korea's economic growth through attaining technology competitiveness. The study is focused on selecting outstanding small and medium corporate laboratories with high R&D capability and fostering the selected laboratories. The main topics of this study are to establish logics regarding the selection process and to propose the implementation schemes of the process. For the selection of outstanding laboratories, this study presents a evaluation indicators based on logic model and verification of the validity of following evaluation indicators through a Pilot test. The evaluation indicators from this study are expected to be in practical use as a reference for support policies of outstanding laboratories' R&D activities. For corporations, these indicators can be used to examine their R&D capability. This study also suggests differentiated policy support measures using the findings to maximize the effectiveness of the selection process.

• Journal of Technology Innovation
• /
• v.9 no.2
• /
• pp.17-33
• /
• 2001

In the knowledge-based economy, technological knowledge (TK) is reckoned key subject of knowledge management. Despite growing recognition, it has long been considered an intractable task to develop precise measures of TK and, as a remedy, a number of R&D-related proxy indicators have been employed. Although voluminous previous research has examined the structure and process of technological innovation by using proxy indicators, the inquiry into the relationship among respective indicators has remained unexplored. In this research, we take three most frequent proxy indicators of TK, R&D human resources, R&D stock, and patents, and investigate the correlation among respective measures. In addition, the dynamic pattern of time lag between technological input and output is also analyzed.

• Journal of Digital Convergence
• /
• v.15 no.5
• /
• pp.151-159
• /
• 2017

This paper proposes various policies for improving the process of performance analysis on R&D expenditure of the public funds in the Korean ICT industry. In order to do so, this paper figures out the present issues relating to performance indicators, methodologies for performance evaluation, release and comparison of evaluation results, follow-up study, integrated management system for performance analysis in ICT industry. In this paper, we suggest some new policies such as consulting an expert for improving performance indicators, maintaining the pre-determined performance indicators, introducing the qualitative performance indicators for measuring effectiveness, adopting the new methodologies reflecting R&D characteristics, introducing the new performance index for comparing annual performance, requiring follow-up study for major ICT R&D, introducing the independent and temporary organization for R&D performance analysis in ICT industry. In conclusion, these new policies will raise effectiveness and efficiency of the government R&D expenditure in ICT industry as well as strengthen the industry competitiveness.