• Journal of Information Technology Applications and Management
• /
• 제13권3호
• /
• pp.145-179
• /
• 2006

The measurement of informatization program effectiveness continues to be a central concern of both academics and practitioners. However, most studies of it have been inclined too heavily toward testing validity and reliability of evaluation measure to strengthen the theoretical rigor. Drawbacks of this approach have been pointed out as being unsuitable for evaluating the effectiveness of informatization program due to lack of practical relevance. The paper comprises four sections to solve the problem. 1) Through literature review of theoretical conceptual models in IS effectiveness and the domestic and foreign cases on informatization program, we establish the evaluation framework which consists of evaluation phases, 6 evaluation areas, and 22 evaluation items, and developed evaluation measures of each items. 2) Different researchers have addressed different aspects of attribute of measure. With comparing them to each other, 3 mutually exclusive criteria on coincidence analysis(usability of measurement. reliability of measurement. and validity of content) are identified. 3) Data are gathered from respondents in informatizaiton program-related organizations such as policy agency, management agency, support agency, and recipient agency. 4) The degree of coincidence of each evaluation measure are analyzed by course of an coincidence analysis procedure, which includes the reliability test, arithmetical test, and criteria validity test. The academic implication of this study is that researchers can gain an insight into the coincidence analysis to approach the problem of deciding on an indicator of performance for informatization program. There is also practical implication for applying the potential means of improving the practical relevance of conducting evaluating the informatization program.