• 한국벤처창업학회:학술대회논문집
• /
• 2009.08a
• /
• pp.5-24
• /
• 2009

This study is the result of meta evaluation for the self evaluation of defense R&D programs in Korea. The overall meta evaluation result of defense R&D programs gained 74.3 points out of 100, so it was evaluated as 'Good'. But it demonstrated that further improvement for overall system of defense R&D programs evaluation is required. And especially, it demonstrated that more alternatives are necessary in order to improve the utilizations and the feedbacks of evaluation results. The evaluation context component gained 80.2 points out of 100, so it was evaluated as 'Very Good'. The evaluation input component gained 73.1 points out of 100, so it was evaluated as 'Good'. The evaluation process component gained 74.8 points out of 100, so it was evaluated as 'Good'. From the analysis, it is attempted to identify possible problems and to find out the ways of improvements related to the self evaluation system of defense R&D Programs.

• Journal of Korean Society of Industrial and Systems Engineering
• /
• v.32 no.3
• /
• pp.59-70
• /
• 2009

This study is the result of meta evaluation for the self evaluation of defense R&D programs in Korea by using meta evaluating indicators. The overall meta evaluation result of defense R&D programs gained 74.3 points out of 100, so it was evaluated as 'Good'. But it demonstrated that further improvement for overall system of defense R&D programs evaluation is required. And especially, it demonstrated that more alternatives are necessary in order to improve the utilizations and the feedbacks of evaluation results. The evaluation context component gained 80.2 points out of 100, so it was evaluated as 'Very Good'. The evaluation input component gained 73.1 points out of 100, so it was evaluated as 'Good'. The evaluation process component gained 74.8 points out of 100, so it was evaluated as 'Good'. And the evaluation outcome component gained 69.0 points out of 100, so it was evaluated as 'Good'. Basic model of meta evaluation was derived from the literature review and brain storming. And this meta evaluation model was determined by adopting the result of experts who performed evaluations for defense R&D programs in recent years. The reliability of components and items was verified by Cronbach's a coefficient. It was over 0.6 in evaluation components and items. And the reliability of evaluation context was 0.877, that of evaluation input was 0.755, that of evaluation process was 0.755, that of evaluation output was 0.755 respectively. From the analysis, it is attempted to identify possible problems and to find out the ways of improvements related to the self evaluation system of defense R&D programs. The ultimate objective of this study is to manage the programs effectively and improve the reliability and the objectiveness of the defense R&D programs.

• Journal of Information Technology Applications and Management
• /
• v.15 no.1
• /
• pp.203-224
• /
• 2008

The purpose of this study is to investigate the evaluation perspectives of IT business value and empirically analyze the differences of evaluation perspectives between ex ante evaluation and post implementation evaluation, and among various information system types. 4 evaluation perspectives which have been used in the previous studies and in the field of industry practices are financial, customer, internal process, and growth, which are based on the BSC model. Data of 98 information systems have been collected from 89 companies including manufacturers, banks, insurances, and stock trading companies. As results of multi-way MANOVA test, 3 out of 8 hypothesis have been accepted statistically. While the order of importance of 4 perspectives in ex ante evaluation is customer, financial, internal process, and growth, the order of post implementation evaluation appears to be much different : that is, internal process, customer, financial, and growth. These findings provide insights for both IT practitioners and researchers.

• Journal of Information Technology Services
• /
• v.17 no.1
• /
• pp.59-77
• /
• 2018

Software process (SP) authentication is a certification system designed in order to improve software process and ensure software reliability of the Korean small and medium enterprises (SMEs). It is composed of 3 grades, 17 evaluation items and 70 detailed evaluation items (software process). In order to certificate grade 2, 42 detailed evaluation items must be satisfied. In order to certificate grade 3, all 70 detailed evaluation items must be satisfied. SMEs are more vulnerable to manpower and material resources than the major company. Therefore, it is difficult for SMEs to invest in the long term to improve the software process. If we can provide the core evaluation items to SMEs, SMEs will be able to improve software quality more efficiently and effectively. It can be used as a short way to acquire SP authentication. The evaluation items of SP are defined as influencing each other. Therefore, the core evaluation items might be established using related techniques. Although we searched for previous researches and related informations, the associated study was insufficient. In this study, ANP (Analytic Network Process) technique was applied, and the core evaluation items of SP authentication were established. SMEs will be able to acquire SP authentication more easily using the core evaluation items. Finally, we expect that this study will be used as a basis material for creating the SP 2.0 version.

• Journal of Korean Society of Industrial and Systems Engineering
• /
• v.28 no.3
• /
• pp.121-132
• /
• 2005

Nowadays, Information Technology(IT) has become one of the most critical managerial decision areas, because its role in the business practice tends to be more important and hence the amount of the investment in growing. Since there is no dominant and promising measure to evaluate the IT performance, the managements have difficulties on the decision process for IT investment. In this paper, we propose an IT performance evaluation model. The proposed model starts with finding the business performance evaluation areas and the corresponding KPIs, and then concentrates on the IT performance evaluation areas corresponding to the business performance evaluation areas. To extract the specific and reliable KPIs for the IT performance measure, we map the business performance evaluation areas into the corresponding IT performance evaluation areas and then combine and adjust these KPIs with the KPIs obtained from the experiential field interview. Finally, We present the complete model which has 11 business performance indices and 48 IT performance indices are completed.

• 한국디지털정책학회:학술대회논문집
• /
• 2006.06a
• /
• pp.87-109
• /
• 2006

The purpose of this paper is to examine the present performance evaluation methods and to make Monte Carlo Simulation Model for the IT-based Government innovation project. It is suggested the proper ways in applying of Monte Carlo Simulation Model by integration of present evaluation methods. It develops the theoretical framework for this paper, examining the existing literature on proposing an approach to the key concepts of the economic impact analysis methods. It examines the actual conditions of performance evaluation focusing on the It-based Government Innovation project. It considers how the simulation model is applied to the performance management in the public innovation project focusing on the framework, process and procedure of performance management.

• Journal of Information Technology Services
• /
• v.8 no.4
• /
• pp.141-150
• /
• 2009

There are no objective standards for selection and improvement of SLA evaluation metrics for IT service. In this study, we analyze the current IT maturity models for selection and improvement of the metrics and then we derive them according to the maturity levels and propose the redesigned maturity model. To verify whether the model is applicable, we execute a case study based on the D company. We apply the proposed evaluation metrics of the maturity models to the D company and evaluate the metrics. We select a proper level of the D company and an improvement line after measuring evaluation metrics in the maturity level 2. We propose improvement guidelines of evaluation metrics which score is less than the improvement line's and derive SLA evaluation metrics. By using the SLA evaluation metrics for a year, we prove that the way of selection and improvement is useful.

• Korean Institute of Interior Design Journal
• /
• no.40
• /
• pp.117-125
• /
• 2003

If the collection of exhibit objects as well as exhibit planning are not in accord with the overall purpose of the exhibit, it will be difficult to attain satisfactory results. The present study focuses on the exhibit evaluation. The exhibit evaluation is very important because it allows the whole process of exhibit development to be reviewed, and it can point a way towards improving the exhibit. Research was conducted by literature review for the museum evaluation methods and by survey for the status of Korean exhibit evaluation. Several Korean History Museum Exhibitions were targeted as cases for this study. Structured and open-ended interviews with 70 museum specialists including curators and exhibit designers were conducted. In conclusion, museum evaluation methods were timely arranged first; front end analysis, second; formative evaluation, third; summative evaluation. Through the survey, one forth of museum specialist didn't have the concept of the exhibit evaluation. However they felt the importance of it.

• Journal of Information Technology Services
• /
• v.3 no.1
• /
• pp.117-126
• /
• 2004

In this study an evaluation method is proposed, with the strong relationship between the ante and post audit works evaluation. The ante evaluation does in general and systematic ways, with past audit results and proposals. The criteria-based evaluation is used for the audit activities and outputs. The critical success factors for the projects applies to the activities evaluation and product evaluation framework to the output. The ante and post evaluation have the cross evaluation items to be the whole and continuous one. Eighteen common evaluation items are derived, with their own evaluation elements in detail, which have only differences of the actually done and expectations between the ante and post. The proposed evaluation makes the ante and post one to be related with each other in a systematic and objective ways, to enhance the reliability of audit results and effectiveness of the audit.

• Journal of Multimedia Information System
• /
• v.6 no.4
• /
• pp.271-282
• /
• 2019

There is a description that you cannot manage what you do not measure. The Korea Ministry of National Defense (MND) is conducting evaluations in various fields to obtain meaningful effects from IT investments, and views that the evaluation of the defense informatization sector is divided into defense informatization policy evaluation and defense informatization project evaluation. The defense informatization level evaluation can measure the informatization level of MND and the armed forces or organizations. Since the evaluation system being studied to measure the level of defense informatization is composed mainly of qualitative metrics, it is necessary to reconstruct it based on quantitative metrics that can guarantee objectivity. In addition, for managing the level of change by evaluation objects, the evaluation system should be designed with a focus on homeostasis of metrics so that it can be measured periodically. Moreover, metrics need to be promoted in terms of performance against targets. To this end, this study proposes to measure the level of defense informatization by dividing it into defense information network, computer systems, interoperability and standardization, information security, information environment, and information system use, and suggests their metrics.