• Knowledge Management Research
• /
• v.10 no.2
• /
• pp.65-84
• /
• 2009

The purpose of this study is to develop Meta Evaluation Indicators for Defense R&D Programs in Korea. At first, the four components of this meta evaluation model were designed, which are evaluation context, evaluation input, evaluation process and evaluation outcome. And fifty two indicators for this meta evaluation were developed by experts who performed evaluations for Defense R&D Programs. The reliability of components and items was verified by Cronbach's ${\alpha}$ coefficient. It was over 0.6 in all areas. The validity of components and items was verified by Factor Analysis. Analytic Hierarchy Process method was used in assigning the evaluation weight. The survey of twenty two evaluators participated in Defense R&D Programs showed that the Consistency Ratio was under 0.1 in evaluation components and items. In this study, an objective and reasonable set of Defense R&D Meta Evaluation Indicators was developed to increase the responsibility of Defense R&D Programs and improve the quality of evaluation results.

• Journal of Engineering Education Research
• /
• v.16 no.3
• /
• pp.28-41
• /
• 2013

The purpose of this study was to develop the Meta-Evaluation Criteria which can comprehensively and systematically evaluate and improve the Accreditation Evaluation System for Engineering Education. This will provide useful implications for establishing value, direction, and improvement remedy for Engineering Educations Accreditation System of Korea. In order to achieve the purpose of this study, Meta-Evaluation Criteria was developed to comprehensively and systematically assess and analyze the Accreditation Evaluation System of Engineering Education. The research methodology used to study these subjects: literature review, interviews with experts, Delphi survey (three times), survey, analysis of narrative opinion and secondary source. The results and conclusions of this study can be summarized as follows: First, the final Meta-Evaluation Criteria were developed so as to comprehensively and systematically assess and analyze the Accreditation Evaluation System of Engineering Education. The criteria's validity and reliability were also identified. Specific details are as follows: In a draft plan of the Meta-Evaluation Criteria, Meta-Evaluation concept was defined as evaluation on the whole range of Accreditation Evaluation System of Engineering Education. Meta-Evaluation Criteria was designed to be based on a systematic approach and applies the phased approach to the lower component to reflect evaluation's characteristics. Then validity and reliability of the developed draft plan was verified by calculating Content Validity Ratio (CVR), Degree of Consensus, Degree of Convergence and Cronbach's alpha. The final developed Meta-Evaluation Criteria obtaining validity and reliability were composed of 5 evaluation areas (environment, input, performance, result, utilization), 15 evaluation items, 68 evaluation indicators.

• Journal of Korea Technology Innovation Society
• /
• v.12 no.1
• /
• pp.113-136
• /
• 2009

The purpose of this study is to develop meta evaluating indicators for Defense R&D Programs in Korea. At first, the four components of this meta evaluation model were designed, which are evaluation context, evaluation input, evaluation process, evaluation output. And fifty two indicators were developed for this meta evaluation. Basic model of meta evaluation was derived from the literature review and brain storming. And this meta evaluation model was determined by adapting the result of experts who performed evaluations for Defense R&D Programs in recent years. Factor Analysis method was used to verify the validity of meta evaluation model. The survey of fifty three members turned out that Cronbach' ${\alpha}$ was over 0.6 in evaluation components and items. And the reliability of evaluation context was 0.877, that of evaluation input was 0.755, that of evaluation process was 0.755, that of evaluation output was 0.755 respectively. Analytic Hierarchy Process (AHP) method was used in assigning the evaluation weight. The survey of twenty two members showed that the Consistency Ratio was 0.09 in evaluation components and items.

• Journal of Korean Society of Industrial and Systems Engineering
• /
• v.32 no.3
• /
• pp.59-70
• /
• 2009

This study is the result of meta evaluation for the self evaluation of defense R&D programs in Korea by using meta evaluating indicators. The overall meta evaluation result of defense R&D programs gained 74.3 points out of 100, so it was evaluated as 'Good'. But it demonstrated that further improvement for overall system of defense R&D programs evaluation is required. And especially, it demonstrated that more alternatives are necessary in order to improve the utilizations and the feedbacks of evaluation results. The evaluation context component gained 80.2 points out of 100, so it was evaluated as 'Very Good'. The evaluation input component gained 73.1 points out of 100, so it was evaluated as 'Good'. The evaluation process component gained 74.8 points out of 100, so it was evaluated as 'Good'. And the evaluation outcome component gained 69.0 points out of 100, so it was evaluated as 'Good'. Basic model of meta evaluation was derived from the literature review and brain storming. And this meta evaluation model was determined by adopting the result of experts who performed evaluations for defense R&D programs in recent years. The reliability of components and items was verified by Cronbach's a coefficient. It was over 0.6 in evaluation components and items. And the reliability of evaluation context was 0.877, that of evaluation input was 0.755, that of evaluation process was 0.755, that of evaluation output was 0.755 respectively. From the analysis, it is attempted to identify possible problems and to find out the ways of improvements related to the self evaluation system of defense R&D programs. The ultimate objective of this study is to manage the programs effectively and improve the reliability and the objectiveness of the defense R&D programs.

• Journal of Technology Innovation
• /
• v.15 no.1
• /
• pp.87-113
• /
• 2007

For the realization of the national self-defense, the trend of the defense R&D projects becomes global and diverse with the increase of their budget. Thus, for the effective execution of the budget and the successful performance of the projects, the importance of selection, management and evaluation of the R&D projects is emphasized. This study suggests the applicability of a meta-evaluation model for the realization of the successful defense R&D project evaluation system. This study focused on the development of the meta-evaluation model design which can be applicable to the national defense R&D projects. Using the proposed meta-evaluation model, we propose a technique to enhance the fairness and the reliability of the national defense R&D projects evaluation system.

• Journal of Korea Technology Innovation Society
• /
• v.18 no.4
• /
• pp.539-560
• /
• 2015

Korea has enacted the Performance Evaluation law on the R&D Programs since 2006. This study categorizes program size, program history, sustainability, self-evaluation result as antecedents, and analyze the relationship between the antecedents and Meta-evaluation result, the change rate of appropriations. The result shows that program size, program history, self-evaluation result have significant effect to the Meta-evaluation result. And Meta-evaluation result has significant effect to the change rate of appropriations. This study has implication that the performance evaluation system shows significant effect to the appropriations.

• 한국벤처창업학회:학술대회논문집
• /
• 2009.08a
• /
• pp.5-24
• /
• 2009

This study is the result of meta evaluation for the self evaluation of defense R&D programs in Korea. The overall meta evaluation result of defense R&D programs gained 74.3 points out of 100, so it was evaluated as 'Good'. But it demonstrated that further improvement for overall system of defense R&D programs evaluation is required. And especially, it demonstrated that more alternatives are necessary in order to improve the utilizations and the feedbacks of evaluation results. The evaluation context component gained 80.2 points out of 100, so it was evaluated as 'Very Good'. The evaluation input component gained 73.1 points out of 100, so it was evaluated as 'Good'. The evaluation process component gained 74.8 points out of 100, so it was evaluated as 'Good'. From the analysis, it is attempted to identify possible problems and to find out the ways of improvements related to the self evaluation system of defense R&D Programs.

• Journal of Korea Technology Innovation Society
• /
• v.17 no.4
• /
• pp.703-732
• /
• 2014

Meta-evaluation for national R&D programs should be established within the framework of whole R&D system linked with science and technology needs and objectives. The R&D program evaluation system can be understood as a circulation of input, implementation, output, and utilization. In the result of the Delphi survey, which consisted of twenty-four experts (performed in three rounds), twenty-four indices were developed. The meta-evaluation model included sub-items which were important points of view of it as follow (eight items): propriety of planning, sufficiency of information, and propriety of evaluators in evaluation input; appropriateness of method and appropriateness of procedure in evaluation implementation; credibility of output in evaluation output; and usefulness of report and application of evaluation utilization. As the application of the meta-evaluation model, the total mean was lower than 3.0 of the median value of 2.73 (5 point Likert scale). Finally, it was suggested that the feedback of evaluation results should be more enforced to R&D program improvement particularly (evaluation utilization 2.50).

• School Mathematics
• /
• v.13 no.3
• /
• pp.469-484
• /
• 2011

Students experience various affects in solving mathematical problem and learning mathematics. Focusing on meta-affect in affective domain, we explored the types of meta-affect. Our research provides illustrative examples and analysis of meta-affect during solving problem. As a result, meta-affect has four types i.e. monitoring of affect, evaluation of emotion, control of emotion, and utilization of affect. And meta-affect is a main key to decide how to handle affect and influence student's cognitive strategies and affect.

• The Mathematical Education
• /
• v.58 no.4
• /
• pp.519-530
• /
• 2019

According to previous studies, meta-affect based on the interaction between cognitive and affective elements in mathematics learning activities maintains a close mechanical relationship with the learner's mathematical ability in a similar way to meta-cognition. In this study, in order to grasp these characteristics phenomenologically, small group problem-solving cases of 5th grade elementary mathematically gifted children were analyzed from a meta-affective perspective. As a result, the two types of problem-solving cases of mathematically gifted children were relatively frequent in the types of meta-affect in which cognitive element related to the cognitive characteristics of mathematically gifted children appeared first. Meta-affects were actively acted as the meta-function of evaluation and attitude types. In the case of successful problem-solving, it was largely biased by the meta-function of evaluation type. In the case of unsuccessful problem-solving, it was largely biased by the meta-function of the monitoring type. It could be seen that the cognitive and affective characteristics of mathematically gifted children appear in problem solving activities through meta-affective activities. In particular, it was found that the affective competence of the problem solver acted on problem-solving activities by meta-affect in the form of emotion or attitude. The meta-affecive characteristics of mathematically gifted children and their working principles will provide implications in terms of emotions and attitudes related to mathematics learning.