An Analysis of the Factors Affecting User Satisfaction in Computational Science and Engineering Platforms: A Case Study of EDISON

계산과학공학플랫폼 품질 특성이 사용자 만족도에 영향을 미치는 요인에 관한 연구

Abstract

Computational Science and Engineering is a convergence study that understands and solves complex problems such as science, engineering, and social phenomena through modeling using computing resources. Computational science and engineering combines algorithms, computational and informatics, and infrastructure. The importance of computational science is increasing with the improvement of computer performance and the development of large data processing technology. In Korea, Korea Institute of Science and Technology Information (KISTI) has been developing national computational science engineering software and utilization technology by combining basic science and computing technology through EDISON project. The EDISON project builds an open EDISON platform and integrates and services information systems in seven areas of computational science and engineering (computational thermal fluids, nanophysics, computational chemistry, structural dynamics, computational design, and computational medicine). Using this, we have established a web-based curriculum to lay the groundwork for fostering scientific talent and commercializing computational science and engineering software. The purpose of this study is to derive the quality characteristic factors of computational science platform and to empirically examine the effect on user satisfaction. This paper examines how the quality characteristics of information systems, the computational science engineering platform, affect the user satisfaction by modifying the research questions according to the propensity of the computational science platform by referring to the success factors of DeLone and McLean's information system. Based on the results of this study, we will suggest strategic implications for platform improvement by searching the priority of quality characteristics of computational science platform.

1. Introduction

As computational performance has been developed, the use of computational science in physics, chemistry, materials, and life science has attracted attention. Computational Science and Engineering computational science and engineering is an interdisciplinary study that understands and solves complex problems, such as science, engineering, and social phenomena, by modeling using computational resources [1]. computational science and engineering combines and utilizes the following: 1. Algorithms and simulation software; 2. Computer and information science to develop hardware (HW), software (SW), and data management technologies; and 3. Infrastructure, such as supercomputers and visualization equipment. The rapid development of computer performance and communication has provided the basis for the universalization of simulations which uses models and data in the field of science and engineering, thereby increasing the importance of computational science and engineering.

The Korea Institute of Science and Technology Information (KISTI) has been promoting the latest science and technology adaptations to science and technology university students and has positively affected science and engineering education and research. The Education-research Integration through Simulation On the Net (EDISON) project was developed to put computational science and engineering simulation software into practical use. Grasping the quality characteristics of the EDISON platform and knowing how the quality characteristics affect actual user satisfaction are necessary to efficiently use this platform.

In this study, we have reconstructed a successful information system model to make it appropriate for the computational science and engineering(EDISON) platform. We then conducted an empirical study that focused on EDISON platform users who are actually served through the EDISON Project. We used the statistical analysis program SPSS to analyze and interpret the results of the survey. Based on these findings, we developed our proposals and presented directions for future research.

 

2. Theoretical background

2.1 EDISON Project

The ongoing EDISON project at KISTI combines basic science and computing technology for education and research in the science and engineering fields. The project develops national SW and applies it to the curriculum and provides a service. The purpose of the project is to develop science talents and to lay the foundation for commercialization base of computational science and engineering sw.

The main contents of the EDISON project are as follows. First, it did development, dissemination, and an open-source of open-computational science and engineering hub or platform(www.edison. re.kr). It is to develop core technologies and platforms which are web-based computational science and engineering framework for expanding to multi-disciplinary. Second, the project aims to develop computational science and engineering SW which specialized application fields to utilize it in educational, and research fields. It is to develop computational science and engineering SW and contents for education and research in a field of application of science and engineering which integrates basic science and information and communications technology (ICT). Finally, user services are implemented for each specialized applications based on the computational science and engineering platform.

 

(Figure 1) EDISON Main Page(www.edison.re.kr)

 

The EDISON project has built an open EDISON platform and integrates the information system and provide services of seven computational science and engineering fields (computational fluids, nanophysics, computational chemistry, structural dynamics, computational design, computational medicine, and urban environment). As of June 2018, the EDISON platform had approximately 549 types of educational SW and 760 types of content that were being used by approximately 60,000 people. Since 2011, the project has progressed to four stages. In the fourth stage, EDISON aims to build a more user-friendly computational science and engineering platform and to strengthen the ecosystem. In particular, the sophistication of the computational and data integration platform, the development of pre-post processes and CX-Learning content, and the support of a virtuous cycle system are being progressed.

The EDISON service is comprised of the EDISON platform, which provides functions such as web portal configuration and management of computational resources and management of science applications. The simulation software registered in the EDISON platform is defined as a' science application'. The EDISON platform provides the Science Appstore framework, which is a meta-information management system for science application services, user authentication, file input and output services, computing resources, and task management functions. In addition, the EDISON Web portal classifies users as general users, developers, tutors, and students and provides virtual labs and related content for simulation selection, monitoring, result Web visualization, and class utilization. The EDISON Web portal also provides a science workflow environment in which the order of the tasks of several simulation software packages can be determined and executed sequentially or repeatedly. [2] Figure 2 provides a representation of the overall system architecture of the EDISON system process. The EDISON platform can be divided into infrastructure, which supports analysis simulation; EDISON middleware, which manages computing resources and distributes appropriate nodes; and Application Framework, which executes analysis algorithms. In order to execute computational science and engineering simulations, the EDISON platform provides a service framework that can access computational nodes through middleware API to execute analytical simulation. [3]

 

(Figure 2) EDISON Platform Components [4]

 

2.1 EDISON Project (9.5pt)

In this study, previous studies on the quality factors of the information systems were investigated. A lot of research has been done on use-quality factors in the field of information systems.

J.Y. Lee and E.J. Lee analyzed the impact of university e-learning courses on learners' satisfaction on system, information, and service qualities. In addition, J.Y. Lee and E.J. Lee checked whether the influence of quality factors differs according to subject series and number of students in the e-learning class. As a result, information, service, and system qualities were shown to influence learning satisfaction, with information quality having the strongest influence on learner satisfaction. [5] B.G. Kim and I.K. Yoon studied the factors that influence user satisfaction and continuous intention through the quality characteristics of a SNS. The system quality of a SNS uses security, efficiency, safety, ease of use, and immediate connectivity variables.

Service quality uses research variables, such as responsiveness, empathy, confidence, and service diversity, to study factors affecting user satisfaction and continuous use intention. [6] H.Y. Jang and M. J. Noh analyzed the effects of the Internet banking channel's information system quality on the user satisfaction and service loyalty of account inquiry and money transfer services. The Internet banking channel's information system quality was largely measured using three variables: information quality (accuracy, comprehension), service quality (responsiveness, reliability), and system quality. [7]Most studies have been conducted based on DeLone and McLean (1992), which structured the information system success model by analyzing more than 180 studies on the factors that affect the success of information systems. DeLone and McLean divided information system success factors into six categories. This study divides the factors that affect the success of an information system into six domains-system quality, information quality, intention to use, user satisfaction, personal influence, and organizational influence-to structure the Information System (IS) Success Model. Also, it important to note is that a detailed description is provided for each quality factor on how it affects the success of an information system.

DeLone and McLean showed that a more appropriate information system for user purposes results in greater user satisfaction and greater performance of the individual that, in turn, increase the organization's performance. [8] Therefore, user satisfaction is used as a representative variable to measure the performance of the information system. This approach structured the factors that influence the success of the information system and established the criteria to measure this success. Meanwhile, researchers analyzed more than 150 papers published in middle scholarly journals that cited the existing IS Success Model [8] and reconstructed it, as shown in Figure 3.

This study conducted a survey based on the definitions in DeLone and McLean to determine whether the quality factors of the computational science and engineering platform affect user satisfaction.

System quality represents the desirable characteristics of an information system, and ease of use, system flexibility, system reliability and learning usability, intuition, sophistication, and response time are its measures.

 

(Figure 3) Reconstructed Information System Success Model [8]

 

In particular, the quality of the system was replaced with the quality of the simulated SW that was serviced on the computational science and engineering platform. Information quality was set as a measure of relevance, understandability, accuracy, consistency, completeness, usability, and others as the desirable characteristics of the system results. For information quality, the EDISON platform replaced the simulated SW with the quality of the contents containing information that enables smooth utilization. Service quality represents the quality of the support that system users receive from the information systems organization and IT support personnel.

Responsibility, accuracy, reliability, and technical capabilities are its measures. User satisfaction was measured as user satisfaction with reports, websites, and support services. Table 1 describes the questions on the survey based on this content.

 

3. Research Model and Analysis Method

3.1 Research Model

In this study, system quality is selected as the evaluation index for running the simulation on the actual platform. For information quality, the quality of the SW information, such as manuals, lecture materials, and other information that provides instructions on how to use the platform, is selected as the evaluation index. For quality of service, the reliability and accuracy of responses to requests and inquiries are selected as the evaluation indexIn this research, we study user satisfaction with the platform by setting system, information, and service qualities as research variables—as in previous research. Figure 4 illustrates the research model for this study.

 

(Figure 4) Research Model

 

This study conducted a survey based on the definitions in DeLone and McLean to determine whether the quality factors of the computational science and engineering platform affect user satisfaction.

System quality represents the desirable characteristics of an information system, and ease of use, system flexibility, system reliability and learning usability, intuition, sophistication, and response time are its measures. In particular, the quality of the system was replaced with the quality of the simulated SW that was serviced on the computational science and engineering platform. Information quality was set as a measure of relevance, understandability, accuracy, consistency, completeness, usability, and others as the desirable characteristics of the system results. For information quality, the EDISON platform replaced the simulated SW with the quality of the contents containing information that enables smooth utilization. Service quality represents the quality of the support that system users receive from the information systems organization and IT support personnel. Responsibility, accuracy, reliability, and technical capabilities are its measures. User satisfaction was measured as user satisfaction with reports, websites, and support services. Table 1 describes the questions on the survey based on this content.

 

3.2 Analysis Method

The questionnaire for this study was administered twice to participants of the 7th EDISON SW Utilization Contest held on February 23, 2018. Of the total 244 attendees, 185 were recalled, and the analysis was conducted on a total of 164 cases that excluded missing values.

 

(Table 1) Survey questions by research model factors Factors Survey details

 

4. Empirical Analysis Results

4.1 Sample Characteristics

Of the 164 cases, 127 (77.4%) were male, and 37 (22.6%) were female. Regarding their field of usage, 26 (15.9%) were in nanophysics, 25 (15.2%) were in computational chemistry, 28 (17.1%) were in structural dynamics, and 50 (30.5%)

were in computational design, 20 (12.2%) were in computational medicine, and 15 (9.1%) were in urban environment. Most respondents were users with less than six months of EDISON experience (84.2%, 51.2%), 30 (18.3%)

had less than one year of experience, and 35 (21.3%) had less than three years of experience. Fifteen respondents (9.2%) had more than three years of experience.

Factor analysis and Cronbach’s alpha were used to verify the validity and reliability of the measurement tools. The factor analysis is considered to be significant when factor loading is 0.4 or higher and is considered very important when factor loading is 0.5 or higher. The factor loadings of the items used in this study were found to be higher than 0.5, and validity results were obtained except for content quality 3. Therefore, we excluded question 3—representing content quality—from this analysis. In addition, the simulation results showed that the quality of the simulation was divided before and after the simulation. Therefore, in this analysis, the quality before the simulation was labeled simulation quality 1 and the quality after the simulation was labeled simulation quality 2.

Reliability analysis is judged using Cronbach’s alpha, for which a closer value to 1.0 indicates greater reliability. In general, the empirical analysis indicated that the Cronbach’s alpha coefficient was 0.7 or higher [11]. Table 2 shows the results of the reliability analysis for evaluating the internal consistency of each metric. Cronbach’s alpha for content quality was .888, for service quality was .934, for simulation quality 1 was .806, and for simulation quality 2 was .708, indicating values 0.6 or higher.

 

(Table 2) Validity of measurement tools for reliability verification

 

4.2 Empirical Analysis Results

A multiple regression analysis was performed to investigate the effect of simulation, content, and service qualities on user satisfaction for the computational science and engineering platform. To perform the regression analysis, Durbin-Wats on 's autocorrelation analysis of the dependent variable was performed, and the result was close to 2, at 1.638, indicating independence of the dependent variables. As a result, we confirmed that these data are suitable for the regression analysis.

The multiple regression analysis indicates that the R2 value of this study is 0.568, or that the independent variable accounts for 56.8% of the dependent variable. The autocorrelation analysis shows that the Durbin-Watson value was 1.705, which is close to 2, indicating the independence of the dependent variables and no problems with the analysis. The analysis of variance results showed that the F value was 25.195 and that the significance probability was 0.000, which was statistically significant. The regression equation was also significant. Additionally, no correlation between the independent variables was confirmed, and the VIF value for verifying multicollinearity was less than 10.

 

(Table 3) Overview of the model

 

The multiple regression analysis results show that the quality factors of the computational science and engineering platform had a significant effect on user satisfaction (p =0.00 <0.5). Content quality and simulation quality 2, which represent the information quality of the COMPUTATIONAL SCIENCE AND ENGINEERING platform, were found to affect user satisfaction. The EDISON platform's user satisfaction tended to increase as content quality and quality after simulation increased. The effect of information quality on beta quality was 0.564, and that of simulation quality 2 was 0.201. The explanatory power of the quality factor, which explains user satisfaction, was 56.8%

 

(Table 4) Model results

 

5. Conclusion and Further Research

5.1 Conclusion

The purpose of this study is to verify through empirical analysis the quality level of the EDISON platform of the EDISON project using the scale of the information system success factors from DeLone and McLean (2003).

The research shows that information quality and some simulation quality affect user satisfaction when using the EDISON platform. In particular, the influence of information quality was relatively strong. Although a demand survey was used to ensure that the EDISON platform was designed and implemented to meet the needs of actual users, an analysis of user satisfaction from actual users during operations has been lacking. In this study, we obtain a better understanding of the quality characteristics of the EDISON platform and suggest meaningful implications for future platform-quality improvement strategies.

The limitation of this research is that it reflects only the quality characteristics of the success factors of the information system from DeLone and McLean (2003). This research did not measure research characteristics other than the three quality characteristics.

 

5.2 Further Research

Research is lacking on how the technologies in the EDISON platform that are currently being developed and serviced affect actual user satisfaction. In the future, based on the success factors model of the information system, the quality factors of computational science platforms can be derived, and the technology acceptance model can be used to study how these factors affect the usability and ease of users and, as a result, user satisfaction.

Further, based on the information system’s success factors, the analysis on whether the user satisfaction level will affect the reuse intention of the EDISON platform can be conducted, contributing to the establishment of strategies for its continuous service.