This paper empirically examines factors that potentially influence the success of a Web-based semantic search engine. A research model has been proposed that shows the impact of quality-related factors upon the effectiveness of a semantic search engine, based on DeLone and McLean's(2003) information systems success model. An empirical study has been conducted to test hypotheses formulated around the research model, and statistical methods were applied to analyze gathered data and draw conclusions. Implications for academics and practitioners are offered based on the findings of the study. The proposed model includes three quality dimensions of a Web-based semantic search engine-namely, information quality, system quality and service quality. These three dimensions each have measures designed to collectively assess the respective dimension. The model is intended to examine the relationship between measures of these quality dimensions and measures of two dependent constructs, including individuals' net benefit and user satisfaction. Individuals' net benefit was measured by the extent to which the user's information needs were adequately met, whereas user satisfaction was measured by a combination of the perceived satisfaction with search results and the perceived satisfaction with the overall system. A total of 23 hypotheses have been formulated around the model, and a questionnaire survey has been conducted using a functional semantic search website created by KT and Hakia, so as to collect data to validate the model. Copies of a questionnaire form were handed out in person to 160 research associates and employees working in the area of designing and developing semantic search engines. Those who received the form, 148 respondents returned valid responses. The survey form asked respondents to use the given website to answer questions concerning the system. The results of the empirical study have indicated that, of the three quality dimensions, information quality was found to have the strongest association with the effectiveness of a Web-based semantic search engine. This finding is consistent with the observation in the literature that the aspects of the information quality should serve as a basis for evaluating the search outcomes from a semantic search engine. Measures under the information quality dimension that have a positive effect on informational gratification and user satisfaction were found to be recall and currency. Under the system quality dimension, response time and interactivity, were positively related to informational gratification. On the other hand, only one measure under the service quality dimension, reliability was found to have a positive relationship with user satisfaction. The results were based on the seven hypotheses that have been accepted. One may wonder why 15 out of the 23 hypotheses have been rejected and question the theoretical soundness of the model. However, the correlations between independent variables and dependent variables came out to be fairly high. This suggests that the structural equation model yielded results inconsistent with those of coefficient analysis, because the structural equation model intends to examine the relationship among independent variables as well as the relationship between independent variables and dependent variables. The findings offer some useful implications for owners of a semantic search engine, as far as the design and maintenance of the website is concerned. First, the system should be designed to respond to the user's query as fast as possible. Also it should be designed to support the search process by recommending, revising, and choosing a search query, so as to maximize users' interactions with the system. Second, the system should present search results with maximum recall and currency to effectively meet the users' expectations. Third, it should be capable of providing online services in a reliable and trustworthy manner. Finally, effective increase in user satisfaction requires the improvement of quality factors associated with a semantic search engine, which would in turn help increase the informational gratification for users. The proposed model can serve as a useful framework for measuring the success of a Web-based semantic search engine. Applying the search engine success framework to the measurement of search engine effectiveness has the potential to provide an outline of what areas of a semantic search engine needs improvement, in order to better meet information needs of users. Further research will be needed to make this idea a reality.
Introduction The e-government is concerned with using Internet and Web technologies to exchange information and services with citizens, businesses and other related organizations, and it centers on three functions, namely informational, interactive, and transactional [UN, 2001]. Many developed countries like the U.S. have been actively involved in e-government projects, since they enable both more effective public services for citizens and more efficient internal operations. Korea is among these leading countries that are planning to leverage computer and communication technologies to provide for integration of work processes and information as well as convenient access to information and services. For this reason, evaluating e-government projects is becoming a crucial issue for both researchers and policy-makers. However, most research to date has primarily focused on a model of success of an e-government system designed for citizens, overlooking internal systems specifically created for employees working in a public organization. This paper is intended to propose a model to measure the success of a Web-based information system designed for use by internal users at Chungwadae, the executive branch of Korea's central government. The paper is also aimed at applying the model to the assessment of the present system being used at Chungwadae in comparison with the preceding system. Evaluating an e-Government System The most widely cited model of information systems success today is that of DeLone and McLean[1992, 2003, 2004]. The original model states that the success of an information system can be measured using six dimensions, including system quality, information quality, use, user satisfaction, individual impact, and organizational impact. Although the ultimate success of an information system may be reflected in the impact that the system has upon individuals as well as an organization, aspects of using the system such as system use and user satisfaction can play an important role in determining the system success, because the system would be a sheer failure if users don't like and use the system. As a response to criticisms given by numerous researchers, the authors adapted their model to fit the emerging Web-based environment. The revised model[DeLone and McLean, 2003] they offered included an additional quality dimension, namely service quality, and combined individual and organizational impacts into net benefits which can also influence user satisfaction. The e-government system success model can be built around this updated model. Our model incorporates information quality, system quality, and service quality as in the DeLone and McLean model. However, the 'system use' dimension has been replaced by perceived usefulness, as suggested by Seddon[1998]. In addition, because the e-government systems that this paper focuses on are internal public systems used in government agencies, the 'net benefits' dimension has been replaced by perceived work efficiency. Based on the proposed model, a total of nine hypotheses have been formulated which we tested using an empirical analysis. Methods A questionnaire form has been created with items that are designed to examine the relationships among the variables in the model. The questionnaire has been handed over, in person, to 65 members of Chungwadae staff who are now actively using the E-Support System, the present information system created to support internal work at Chungwadae. We made arrangements to meet with each individual who agreed to participate in our survey, and helped to fill out the survey form with explanations. Of the 65 copies that were delivered, only 33 were returned, and 30 responses of these have been adopted for our analysis, since three were not valid. The extremely small sample size was due to the limited number of staff members who had adequate experience required of this study. Results We gathered data from the questionnaire survey and analyzed them using a regression analysis to test the hypotheses. As shown in the table below, the results indicated that all three dimensions of an information system’s quality are positively related to user satisfaction. However, information quality and system quality were found to be positively related to perceived usefulness, while service quality was not. In addition, perceived usefulness is not positively related to user satisfaction, implying that a user may find a system useful, but may not be satisfied with it. Finally, user satisfaction and perceived usefulness both are positively related to perceived work efficiency. This suggests that workers' positive experience with the system is important to guarantee favorable work efficiency. Conclusions We conclude that the proposed model proved useful in measuring the success of an internal information system used by a government agency. To demonstrate the applicability and usefulness of the model in the paper, we applied the model to the assessment of the present internal system used at Chungwadae in comparison with the preceding system. The results showed that the present system outperforms the preceding one in a statistically significant way. Future research will have to focus on applying the model to Korea's governmental agencies other than Chungwadae and examine whether it proves applicable in different types of governmental organizations.