• Journal of Multimedia Information System
• /
• 제7권1호
• /
• pp.73-86
• /
• 2020

The well-known sentence, "You can't manage what you don't measure" suggests the importance of measurement. The Ministry of National Defense (MND) in Korea is measuring the effort of informatization for various dimensions such as validity, adequacy, and effectiveness using the MND evaluation system to obtain positive and significant effects from informatization. MND views the defense informatization domain as divided into the defense informatization policy, the defense informatization project, and the defense informatization level, which can measure the informatization capability of the MND and the armed forces or organizations. Furthermore, it feels there is some limitation, such as those related to ambiguity and reliability, present in the system. To overcome the limitations in the existing system to evaluate the defense informatization policy, this study proposes a revised evaluation framework for the policy of defense informatization, its indicators, and measurement methods.

• Journal of Information Technology Applications and Management
• /
• 제9권3호
• /
• pp.31-46
• /
• 2002

The objective of this paper is to develop an evaluation model for pilot informatization projects. There are relatively many researches on evaluation models for information systems. These previous researches, however, lack of comprehensive view of informatization Projects and pilot systems. We apply and test a hybrid evaluation model for to measure the success of pilot informatization projects. A user satisfaction model has been used and hypotheses are developed to find relationships of evaluation factors. The hypotheses have been tested with 51 user surveys. The result of this research can give an insight for the evaluation of pilot informatization projects.

• Journal of Multimedia Information System
• /
• 제6권4호
• /
• pp.271-282
• /
• 2019

There is a description that you cannot manage what you do not measure. The Korea Ministry of National Defense (MND) is conducting evaluations in various fields to obtain meaningful effects from IT investments, and views that the evaluation of the defense informatization sector is divided into defense informatization policy evaluation and defense informatization project evaluation. The defense informatization level evaluation can measure the informatization level of MND and the armed forces or organizations. Since the evaluation system being studied to measure the level of defense informatization is composed mainly of qualitative metrics, it is necessary to reconstruct it based on quantitative metrics that can guarantee objectivity. In addition, for managing the level of change by evaluation objects, the evaluation system should be designed with a focus on homeostasis of metrics so that it can be measured periodically. Moreover, metrics need to be promoted in terms of performance against targets. To this end, this study proposes to measure the level of defense informatization by dividing it into defense information network, computer systems, interoperability and standardization, information security, information environment, and information system use, and suggests their metrics.

• 한국전자거래학회지
• /
• 제25권2호
• /
• pp.29-48
• /
• 2020

정보화 분야는 상대적으로 투자 성과를 가시화하기 제한되는 특성을 가진 바, 정보화 추진 성과에 대한 보다 정량적이고 체계적인 검증 노력이 요구된다. 이에 정부에서는 정보화평가 제도를 지속적으로 확대하고 강화시켜왔으며, 국방부에서도 이러한 정부 기조에 맞추어 정보화평가 제도를 발전시켜왔다. 이러한 배경 하에 2011년 제정된 국방정보화법에서는 정책평가 제도를 통하여 국방정보화정책에 대한 시행 여부를 점검하고 그 성과를 관리하도록 명시하고 있다. 하지만 국방정보화 정책평가는 법제화 직후 한 차례 시범평가 실시 이후 시행되지 못하고 있어 제도의 정상적 운영과 더불어, 평가제도 개선 노력이 필요한 상태이다. 이에 본 연구에서는 국방정보화 정책평가 제도의 안착과 지속적인 발전을 위하여 '14~'28 국방정보화기본계획서상의 31개 정책에 대한 평가를 실시하였다. 이를 위해 국방정보화평가 수행을 위한 평가 제도를 개선하여 제반여건을 정비하였고, 현재 부재한 정책별 성과목표와 지표를 수립할 수 있도록 지원도구를 개발하였다. 그리고 평가단을 구성하여 정책별 평가를 수행하고, 정책평가 수행과정에서의 문제점과 시사점을 분석함으로써 국방정보화 정책평가제도의 실효성 제고를 위한 정책제언을 제시하였다. 본 연구의 정책평가 수행과정과 제시한 정책제언은 국방정보화 분야 뿐 아니라 공공분야 정보화 추진 정책에 대한 성과 관리 측면에서 유의미한 사례연구가 될 것으로 판단된다.

• 한국IT서비스학회지
• /
• 제6권1호
• /
• pp.101-126
• /
• 2007

This study aims to construct the integrative performance evaluation model of informatization programs. In order to achieve the purpose of research, first of all, the three major theoretical paradigms (technical/functional, economic/financial, and interpretive/alternative paradigms) are comparatively analyzed concerning their philosophical and methodological perspectives. Also, a number of important studies and research model in the area of IS (Information Systems) evaluation are critically reviewed. And then actual problems and limitations on the process of implementing IS performance evaluation are sought through investigating domestic and foreign practices. Based upon theoretical and practical review of informatization evaluation and logical reasoning, the components of ideal performance evaluation scheme for informatization program are derived, and the integrative performance evaluation model which can comprehensively represent the relationship among the components is conceptually proposed. For researchers, this study provides the theoretical basis for reasonably deriving the performance variables of informatization programs and empirically validating the performance of managing informatization programs. And in real world, the results of this study are very likely to contribute to presenting the practical guideline for desirably implementing performance evaluation of informatization programs.

• 한국경영과학회지
• /
• 제32권2호
• /
• pp.123-140
• /
• 2007

This study focuses on establishing theoretically-reasonable framework of evaluation areas and items for the informatization program and analyzing the weights for each area and item by means of the AHP. To obtain the goal of this research, firstly, based on broadly reviewing previous literature and logical reasoning, the evaluation criteria were Identified and conceptualized, which were 3 evaluation phases, 5 evaluation areas, and 22 evaluation items. Secondly, The data used for calculating the weight values for the evaluation criteria were collected from 48 academic and practical experts in the field of the informatization program promotion using internet survey. Thirdly, the relative weights among evaluation phases, evaluation areas, and its items were derived by AHP analysis. The findings of this study are thought not only to be useful as a practical guideline in performing evaluating the informatization program but also to provide significant basis for constructing the theoretical framework of evaluating method and management of the informatization program.

• Journal of Multimedia Information System
• /
• 제5권4호
• /
• pp.291-297
• /
• 2018

Performance measurement is not an easy task. However, it is necessary to measure the expected value of the informatization project to be invested, to help the successful progress of the informatization project, and to confirm the result of the informatization project. This study describes the evaluation systems of the defense informatization project in Korea used after the end of the informatization project (post-implementation evaluation stage). In the post-implementation evaluation stage of proposed system, the informatization project is evaluated in terms of performance achievement, project plan compliance, project management, and economic validity. For each element, this study suggests metrics and practical measurement method. With metrics and their method, it can be evaluated which project is better than others. Moreover, the lessons learned can be developed. Under scarce budget condition, defense organizations can make better performance from IT project with the evaluation system suggested in this study.

• Journal of Information Technology Applications and Management
• /
• 제13권3호
• /
• pp.145-179
• /
• 2006

The measurement of informatization program effectiveness continues to be a central concern of both academics and practitioners. However, most studies of it have been inclined too heavily toward testing validity and reliability of evaluation measure to strengthen the theoretical rigor. Drawbacks of this approach have been pointed out as being unsuitable for evaluating the effectiveness of informatization program due to lack of practical relevance. The paper comprises four sections to solve the problem. 1) Through literature review of theoretical conceptual models in IS effectiveness and the domestic and foreign cases on informatization program, we establish the evaluation framework which consists of evaluation phases, 6 evaluation areas, and 22 evaluation items, and developed evaluation measures of each items. 2) Different researchers have addressed different aspects of attribute of measure. With comparing them to each other, 3 mutually exclusive criteria on coincidence analysis(usability of measurement. reliability of measurement. and validity of content) are identified. 3) Data are gathered from respondents in informatizaiton program-related organizations such as policy agency, management agency, support agency, and recipient agency. 4) The degree of coincidence of each evaluation measure are analyzed by course of an coincidence analysis procedure, which includes the reliability test, arithmetical test, and criteria validity test. The academic implication of this study is that researchers can gain an insight into the coincidence analysis to approach the problem of deciding on an indicator of performance for informatization program. There is also practical implication for applying the potential means of improving the practical relevance of conducting evaluating the informatization program.

• 한국항행학회논문지
• /
• 제12권2호
• /
• pp.170-179
• /
• 2008

본 연구는 기존의 건설기업 정보화 수준평가 관련 연구에서 나타나고 있는 연구의 한계점과 건설기업 정보화 특성을 고려하여 건설기업의 경영목표에 따라 정보화 수준을 측정할 수 있는 모형을 설계하였다. 세부적으로 건설기업의 업무특성과 새로운 건설 정보화 수준 평가모형 개발을 위한 요구사항을 정리하고, 이러한 요구사항을 만족시키기 위한 정보화 수준 평가모형의 구성요소 및 측정방법을 설계하였다. 개발된 건설기업 정보화 수준모형은 전문가 집단을 대상으로 델파이 방법과 함께 국내 200개 건설기업을 대상으로 설문조사를 실시하여 통계적 방법을 통하여 정보화 수준평가 모형의 적합성(신뢰성 및 타당성)을 검증하였다. 본 연구의 결과는 건설기업의 경영목표를 개선하기 위한 정보화 추진전략을 수립하는데 활용될 수 있을 것으로 기대된다.

• 산업공학
• /
• 제14권4호
• /
• pp.421-428
• /
• 2001

The economic effects of informatization are differently occurred according to industry types. Thus an evaluation methodology which reflects the industry type characteristics of informatization is needed. In this paper, we develop and analyze the indices for evaluating economic effects through informatization according to industry types, such as manufacturing, construction, financial services and marketing services. The factors related to performance of enterprise informatization can be defined as cost factors, effect factors, risk factors and influence factors. Effect factors are defined as quantitative or qualitative performance of informatization and classified into operational effect factors and strategic effect factors. The influence factor is defined as characteristics of informatization for industry types, that is, value chain of business and business products or services. Economic effect indices are classified according to the activity of business value chain. Economic effects indices of this research can be expected to play a role of general and standardized factors for economic evaluation of industry informatization.