• Journal of Multimedia Information System
• /
• v.7 no.1
• /
• pp.73-86
• /
• 2020

The well-known sentence, "You can't manage what you don't measure" suggests the importance of measurement. The Ministry of National Defense (MND) in Korea is measuring the effort of informatization for various dimensions such as validity, adequacy, and effectiveness using the MND evaluation system to obtain positive and significant effects from informatization. MND views the defense informatization domain as divided into the defense informatization policy, the defense informatization project, and the defense informatization level, which can measure the informatization capability of the MND and the armed forces or organizations. Furthermore, it feels there is some limitation, such as those related to ambiguity and reliability, present in the system. To overcome the limitations in the existing system to evaluate the defense informatization policy, this study proposes a revised evaluation framework for the policy of defense informatization, its indicators, and measurement methods.

• Journal of Information Technology Applications and Management
• /
• v.9 no.3
• /
• pp.31-46
• /
• 2002

The objective of this paper is to develop an evaluation model for pilot informatization projects. There are relatively many researches on evaluation models for information systems. These previous researches, however, lack of comprehensive view of informatization Projects and pilot systems. We apply and test a hybrid evaluation model for to measure the success of pilot informatization projects. A user satisfaction model has been used and hypotheses are developed to find relationships of evaluation factors. The hypotheses have been tested with 51 user surveys. The result of this research can give an insight for the evaluation of pilot informatization projects.

• Journal of Multimedia Information System
• /
• v.6 no.4
• /
• pp.271-282
• /
• 2019

There is a description that you cannot manage what you do not measure. The Korea Ministry of National Defense (MND) is conducting evaluations in various fields to obtain meaningful effects from IT investments, and views that the evaluation of the defense informatization sector is divided into defense informatization policy evaluation and defense informatization project evaluation. The defense informatization level evaluation can measure the informatization level of MND and the armed forces or organizations. Since the evaluation system being studied to measure the level of defense informatization is composed mainly of qualitative metrics, it is necessary to reconstruct it based on quantitative metrics that can guarantee objectivity. In addition, for managing the level of change by evaluation objects, the evaluation system should be designed with a focus on homeostasis of metrics so that it can be measured periodically. Moreover, metrics need to be promoted in terms of performance against targets. To this end, this study proposes to measure the level of defense informatization by dividing it into defense information network, computer systems, interoperability and standardization, information security, information environment, and information system use, and suggests their metrics.

• The Journal of Society for e-Business Studies
• /
• v.25 no.2
• /
• pp.29-48
• /
• 2020

For the visualization of outcome is relatively limited in informatization filed especially, systematic and quantifiable performance management of IT projects and policies is needed. Thus, the government developed its informatization evaluation system and has reinforced it. And Ministry of National Defense (MND) continues to strive for the settlement of the informatization performance evaluation system as well. According to Defense Informatization Law which was legislated in 2011, informatization policies should be assessed annually through informatization policy evaluation system in terms of their enforcement and outcome. However, informatization policy evaluation has not carried out since its pilot enforcement just after the legislation of the law. Hence, we conducted informatization policy evaluation aimed at performance investigation of 31 policies in '14~'18 Defense Informatization Master Plan. We sophisticated the current informatization policy system and we provide guidelines and tools to support the development of performance goal and indicators for each of the policies. Then, the policies were assessed by the evaluation committee we organized for our study, and we analyzed the problems we tackled in the whole process of evaluation and provided proposals for effectiveness enhancement of defense informatization evaluation system. The proposals will be meaningful for performance management in defense informatization sector and in public informatization sector as well.

• Journal of Information Technology Services
• /
• v.6 no.1
• /
• pp.101-126
• /
• 2007

This study aims to construct the integrative performance evaluation model of informatization programs. In order to achieve the purpose of research, first of all, the three major theoretical paradigms (technical/functional, economic/financial, and interpretive/alternative paradigms) are comparatively analyzed concerning their philosophical and methodological perspectives. Also, a number of important studies and research model in the area of IS (Information Systems) evaluation are critically reviewed. And then actual problems and limitations on the process of implementing IS performance evaluation are sought through investigating domestic and foreign practices. Based upon theoretical and practical review of informatization evaluation and logical reasoning, the components of ideal performance evaluation scheme for informatization program are derived, and the integrative performance evaluation model which can comprehensively represent the relationship among the components is conceptually proposed. For researchers, this study provides the theoretical basis for reasonably deriving the performance variables of informatization programs and empirically validating the performance of managing informatization programs. And in real world, the results of this study are very likely to contribute to presenting the practical guideline for desirably implementing performance evaluation of informatization programs.

• Journal of the Korean Operations Research and Management Science Society
• /
• v.32 no.2
• /
• pp.123-140
• /
• 2007

This study focuses on establishing theoretically-reasonable framework of evaluation areas and items for the informatization program and analyzing the weights for each area and item by means of the AHP. To obtain the goal of this research, firstly, based on broadly reviewing previous literature and logical reasoning, the evaluation criteria were Identified and conceptualized, which were 3 evaluation phases, 5 evaluation areas, and 22 evaluation items. Secondly, The data used for calculating the weight values for the evaluation criteria were collected from 48 academic and practical experts in the field of the informatization program promotion using internet survey. Thirdly, the relative weights among evaluation phases, evaluation areas, and its items were derived by AHP analysis. The findings of this study are thought not only to be useful as a practical guideline in performing evaluating the informatization program but also to provide significant basis for constructing the theoretical framework of evaluating method and management of the informatization program.

• Journal of Multimedia Information System
• /
• v.5 no.4
• /
• pp.291-297
• /
• 2018

Performance measurement is not an easy task. However, it is necessary to measure the expected value of the informatization project to be invested, to help the successful progress of the informatization project, and to confirm the result of the informatization project. This study describes the evaluation systems of the defense informatization project in Korea used after the end of the informatization project (post-implementation evaluation stage). In the post-implementation evaluation stage of proposed system, the informatization project is evaluated in terms of performance achievement, project plan compliance, project management, and economic validity. For each element, this study suggests metrics and practical measurement method. With metrics and their method, it can be evaluated which project is better than others. Moreover, the lessons learned can be developed. Under scarce budget condition, defense organizations can make better performance from IT project with the evaluation system suggested in this study.

• Journal of Information Technology Applications and Management
• /
• v.13 no.3
• /
• pp.145-179
• /
• 2006

The measurement of informatization program effectiveness continues to be a central concern of both academics and practitioners. However, most studies of it have been inclined too heavily toward testing validity and reliability of evaluation measure to strengthen the theoretical rigor. Drawbacks of this approach have been pointed out as being unsuitable for evaluating the effectiveness of informatization program due to lack of practical relevance. The paper comprises four sections to solve the problem. 1) Through literature review of theoretical conceptual models in IS effectiveness and the domestic and foreign cases on informatization program, we establish the evaluation framework which consists of evaluation phases, 6 evaluation areas, and 22 evaluation items, and developed evaluation measures of each items. 2) Different researchers have addressed different aspects of attribute of measure. With comparing them to each other, 3 mutually exclusive criteria on coincidence analysis(usability of measurement. reliability of measurement. and validity of content) are identified. 3) Data are gathered from respondents in informatizaiton program-related organizations such as policy agency, management agency, support agency, and recipient agency. 4) The degree of coincidence of each evaluation measure are analyzed by course of an coincidence analysis procedure, which includes the reliability test, arithmetical test, and criteria validity test. The academic implication of this study is that researchers can gain an insight into the coincidence analysis to approach the problem of deciding on an indicator of performance for informatization program. There is also practical implication for applying the potential means of improving the practical relevance of conducting evaluating the informatization program.

• IE interfaces
• /
• v.14 no.4
• /
• pp.421-428
• /
• 2001

The economic effects of informatization are differently occurred according to industry types. Thus an evaluation methodology which reflects the industry type characteristics of informatization is needed. In this paper, we develop and analyze the indices for evaluating economic effects through informatization according to industry types, such as manufacturing, construction, financial services and marketing services. The factors related to performance of enterprise informatization can be defined as cost factors, effect factors, risk factors and influence factors. Effect factors are defined as quantitative or qualitative performance of informatization and classified into operational effect factors and strategic effect factors. The influence factor is defined as characteristics of informatization for industry types, that is, value chain of business and business products or services. Economic effect indices are classified according to the activity of business value chain. Economic effects indices of this research can be expected to play a role of general and standardized factors for economic evaluation of industry informatization.

• Journal of Advanced Navigation Technology
• /
• v.12 no.2
• /
• pp.170-179
• /
• 2008

This study designed model that could scientifically and objectively evaluate a level of informatization of construction companies in consideration of informatization characteristics and limitations identified in researches related to the evaluation of informatization level of construction companies, so at to create the e-business environment for the construction companies including the enhancement of informatization consciousness and to improve capacity to carry out informatization. To do this, the study defined requirements to develop new model of construction informatization level and the characteristics of construction companies, and developed components of informatization level model and measuring methods that could meet the requirements based on theories and literature. The developed informatization level model were reviewed for validity and reliability through statistical method by carrying out the questionnaire survey on 200 companies along with an advisory group of experts. As the result of this study, we could suggest the directions for informatization to narrow the gap between the current level of informatization and the managerial goals in construction company.