M-Learning Systems Usage: A Perspective from Students of Higher Educational Institutions in Sri Lanka

  • Received : 2021.04.20
  • Accepted : 2021.07.15
  • Published : 2021.08.30


Mobile devices have become attractive learning devices for education. The digitalization of the higher education system in Sri Lanka by 2020 is part of the government's effort to modernize and enhance the country's overall education system particularly in view of the COVID-19 pandemic. Theoretically, this study contributes to the M-Learning model in higher education institutions via the integration of literature on technology adoption (TAM and UTAUT) with the variables of Perceived Usefulness, Perceived Ease of Use, Attitude, Effort Expectancy, Social Influence, and Facilitating Condition. The attitude towards M-Learning amongst higher education students was gauged via an online questionnaire survey. The convenience sample comprised 344 students from the Advanced Technological Institutes (ATI) in Batticaloa District, Sri Lanka. Descriptive statistics, a measurement, and structural model, and hypotheses testing were used to analyze the derived data. The findings indicate that mobile learning is significantly affected by perceived ease of use, social influence, effort expectancy, and facilitating condition, but negatively affected by attitude and perceived usefulness. The exhaustive literature review revealed that there are very few M-Learning studies related to digital learning in the context of higher education in the Batticaloa district.


1. Introduction

 In recent years, there has been substantial growth in the mobile learning market. Education institutions and corporate users have become more receptive to the adoption of technological components. Technology in the mobile learning industry has played a significant role in enabling students and educators to interact with the upcoming learning opportunities, thus enabling them to have a richer learning experience. M-Learning integrates the techniques and knowledge of the classroom with the flexibility and scalability of advanced mobile technology to create a unique, fruitful, and efficient learning experience (Gikas & Grant, 2013). Millennials who make up most of the higher education students in this era seem to be very receptive to M-Learning. This generation is naturally proficient in using technological devices having been brought up surrounded by computers and the Internet (Tapscott, 1998). Despite its popularity, M-Learning is yet to be adopted widely in higher education institutions (Herrington & Herrington, 2007). For institutions that have adopted the system, there is the question of whether the mobile devices are being properly used for pedagogical purposes (Herrington & Herrington, 2007).

 The success of M-Learning is dependent upon its active and effective usage by students. However, very few studies had examined the adoption of mobile technology for learning from the standpoint of students. Some studies on M-Learning in the context of the Batticaloa district had examined higher education students’ readiness in using M-Learning. But more insight is needed to understand the factors that drive M-Learning adoption in the context of Sri Lanka (Nawaz & Mohamed, 2020). Hence, this current study aims to investigate the factors that drive M-Learning adoption amongst higher education students in the Batticaloa district.

2. Literature Review

 Millennials make up the majority of the current university population. Millennials are those born after the 1980s and were brought up surrounded by computers and the Internet (Tapscott, 1998). According to Junco and Mastrodicasa (2007), millennials use more technology than any earlier generations. There is pervasive usage of computers, digital cameras, smartphones, tablets, and MP3 players, among Millennials. Online activities such as social networking, downloading, messaging, blogging, and podcasting are also rampant among them. A majority of them have social media presence whereby one-in-five have appeared on YouTube (Malikhao & Servaes, 2010).

 Globally, university students are constantly accompanied by their smartphones and tablets even during classroom sessions for personal engagements. A mobile phone is no longer a luxury but rather a ‘necessity’. Millennials not only use online content, but they also produce and share them. Hence, mobile technology for learning can potentially motivate millennials to learn and achieve improved academic performance.

2.1. Mobile Learning

 Today’s students are dubbed as ‘digital natives’. This ‘digital generation’ entails those born during the ‘digital revolution’ period i.e. between 1995 and 2005. This generation of students expects advanced, Internet-based technologies in their learning institutions. But the reality is that most of the existing hardware is in poor condition while teachers mostly do not have much knowledge about computers and technological devices (Jung, 2018). Hence, a change in institutional culture is highly called for to equip the existing and new generation of students to face 21st-century challenges. Personal devices like smartphones, iPods, tablets, and gaming systems can be used to attain learning materials from inside and outside the classroom, a practice that is increasingly employed globally (Norris et al., 2003).

 According to Kukulska-Hulme (2009), ownership of mobile and wireless devices has changed technology-based learning environments. New technological advancements are not only changing learning environments, but also the cultural and societal norms in schools. With the widespread ownership of mobile devices amongst students, educators have acknowledged that M-Learning is more than just a new concept and that it offers various advantages. Mobile devices are no longer limited to making phone calls and text messaging. They now carry speedy Internet-based computer functions, such as recording sounds, pictures, and videos (Bartholomew et al., 2017).

2.2. M-Learning in Higher Educational Institutions

 Nations worldwide are looking at digital inclusion agendas that integrate M-Learning. Digital inclusion is about enabling all people to contribute to and benefit from the digital economy and society. Early childhood digital media access prepares young children for school, expands their prospects for learning, and closes learning gaps. Higher educational institutions are now embracing technology based learning tools with positive results. Learning has now expanded beyond the classroom and school hours, taking place anywhere at any time (Wang & Lam, 2018).

 Researchers such as Khan et al. 2020) and Samsudeen and Mohamed (2019) had investigated mobile device usage among students aged 18–24 years old. Handheld devices are gaining popularity due to the aspiration to achieve global computing capability. Such devices began with PDAs and palm pilots, MP3 players, and iPods which later progressed to iPads and smartphones (Nawaz & Mohamed, 2020; Ambarwati et al., 2020). The Sri Lankan government has been investing billions of dollars annually to improve its educational sector (Soetjipto et al., 2020).

2.3. M-Learning Usage

 The factors that drive M-Learning adoption have been investigated by looking at two contexts i.e. the university (Khan et al., 2020) and higher educational institutions (Samsudeen & Mohamed, 2019). But the adoption and understanding of M-Learning are rather complex. Barriers in the form of performance-related issues, ease of use, cost, social and ethical issues need to be prudently examined to ensure the comprehensive success of mobile learning in higher education institutions.

 Based on the literature review, there are very few studies on the adoption of M-Learning in Advanced Technological Institute (ATI) in the Batticaloa District, particularly in higher education institutions (Samsudeen & Mohamed, 2019). The available studies had only investigated the general perception of students on M-Learning. There are no studies that determined the factors that drive M-Learning usage.

3. Research Model and Hypotheses

 The most notable theoretical models for this purpose are the Technology Acceptance Model (TAM) (Davis, 1989) and its extensions, and the Unified Theory of Acceptance and Use of Technology (UTAUT) (Venkatesh et al., 2003). These models have been employed in many technology based studies including healthcare informatics, banking, and online shopping. Studies rarely employ the initial versions of technology acceptance theories and models. Researchers often modify these models to incorporate additional context-specific elements that would better fit the context of their studies. M-Learning possesses its own characteristics that are different from other IT contexts. Hence, this current study developed a framework specifically for examining the adoption of M-Learning by higher education students.

 Figure 1 shows that the proposed model is made up of seven constructs. The dependent variable is the M-Learning system (UM) while the independent variables are attitude (AT), effort expectancy (EE), facilitating condition (FC), perceived ease of use (PE), perceived usefulness (PU), and social influence (SI).

OTGHEU_2021_v8n8_637_f0001.png 이미지

Figure 1: Proposed Framework

 A number of hypotheses were then established to reflect the constructs under study and their relationships.

3.1. Attitude

 Attitude refers to a set of emotions, beliefs, and behaviors toward a particular object, person, thing, or event. Attitudes are often the result of experience or upbringing, and they can have a powerful influence over behavior (Raza et al., 2018). An attitude could be generally defined as a way a person responds to his or her environment, either positively or negatively. The attitude of students towards M-Learning affects their usage behavior (Sharma et al., 2016). Thus, the hypothesis below is proposed.

H1: Attitude has a significant impact on M-Learning system usage.

3.2. Effort Expectancy

 Effort expectancy is a belief that the use of a particular technology will be easy and effortless. (Venkatesh et al., 2003). This is a key determinant of M-Learning adoption. Wang et al. (2014) had proved that effort expectancy among females has a higher significance in influencing M-Learning adoption (Venkatesh et al., 2003). In the current study, the perception of M-Learning usage is used to measure effort expectancy. Thus, the hypothesis below is proposed:

H2: Effort expectancy has a significant impact on M-Learning system usage.

3.3. Facilitating Conditions

 Facilitating conditions, in general, were defined as the “perceived enablers or barriers in the environment that influence a person’s perception of ease or difficulty of performing a task. Facilitating conditions entail the key factors that drive students to continue with their studies. These conditions include a good Internet connection. Formally, the term is defined as “the degree to which a student believes that an institution laboratory and local area network connections exist to support the use of the M-Learning system” (Ambarwati et al., 2020). Thus, the hypothesis below is proposed:

H3: Facilitating conditions have a significant impact on M-Learning system usage.

3.4. Perceived Ease of Use

 Perceived ease of use is defined as “the degree to which a person believes that using a particular system would be free of effort. Perceived ease of use entails the students’ perception of how much effort is needed to use the M-Learning system. This factor significantly drives the acceptance and usage of the M-Learning system among higher education students (Hamidi & Jahanshaheefard, 2019). Thus, the hypothesis below is proposed:

H4: Perceived ease of use has a significant impact on M-Learning system usage.

3.5. Perceived Usefulness

 Perceived usefulness the degree to which a person believes that using a particular system would enhance his/ her job performance. This construct is especially related to students’ academic performance, behavior, and reliability. Formally, it is defined as “the degree to which a student believes that using mobile learning would enhance their academic performance” (Davis, 1989). Its integration into TAM has a significant effect on mobile learning usage (Hamidi & Jahanshaheefard, 2019). Thus, the hypothesis below is proposed:

H5: Perceived usefulness has a positive influence on M-Learning system usage.

3.6. Social Influence

 Social influence is the process by which an individual’s attitudes, beliefs, or behavior are modified by the presence or action of others. Mobile device usage can be affected by peer influence (Sharma et al., 2016). Social influence in this context is reflected by the social norms and ethical culture in higher education institutions which drive the usage of new technology. Formally, this construct is defined as “the degree to which individuals perceive that others’ belief is important in their usage of mobile learning”. In this current context, “others” refers to the students’ peers. Thus, the hypothesis below is proposed:

H6: Social influence has a significant impact on M-Learning system usage.

3.7. Use of M-Learning System

 M-Learning usage entails the students’ desire or individual behavior to use the system. In addition, it is the efficacy of the students’ actions towards a specific procedure. M-Learning usage system is determined by the user’s attitude, utility, personal values, confidence, and ease of use (Hamidi & Chavoshi, 2018).

4. Methodology

4.1. Sample and Data Collection

 The study sample comprises students in higher educational institutions, specifically Millennials who are adept at technology usage. First, the minimum sample size was determined which is 350. Data collection was subsequently carried out with the aim to reach an equal or slightly lower sample size than required. By employing the intercept survey method, a total of 344 responses were gathered from students in Advanced Technological Institute (ATI) located in the Batticaloa district. The respondents are IT and Accounting students who have used the virtual learning portal developed by the institution. Differences in the system are not an issue because all the students have access to the same platform. Since the virtual learning portal enables multiple device access, students generally have a similar understanding of M-Learning. The respondents consist of Year 1 to Year 4 students, and their participation in this research is voluntary.

4.2. Measurements

 A structured questionnaire was used to collect the needed data. At the beginning of the questionnaire, a definition of M-Learning is provided to ensure that the students’ interpretation of M-Learning is aligned with that of the research. They are then required to answer questions about their demographic data, usage of technology, and perception of M-Learning. Perceptions of M-Learning include the variables of attitude towards M-Learning, effort expectancy, facilitating condition, perceived ease of use, perceived usefulness, and social influence. The items were taken from Cheon et al. (2012) and measured using a 7-point Likert scale (1 = strongly disagree to 7 = strongly agree).

4.3. Respondents’ Profile

 Table 1 tabulates the respondents’ demographic data. A total of 57.56% of the respondents are females and 42.44% are males. This indicates that there are more female students pursuing tertiary education in ATI. A total of 28.2% of the students are from Year 1, 38.37% from Year 2, 19.77% from Year 3, and 13.66% from Year 4. In terms of Internet usage, 1.74% of the students have never used the Internet, 8.14% spend less than 1 hour per day on the Internet, 30.52% spend 1–5 hours per day, 31.10% spend 6–10 hours per day, 25.87% spend 11–15 hours per day, 3.2% spend 16–20 hours per day, and 2.03% spend more than 20 hours per day. Next, a total of 80.81% of the students own smartphones, 8.43% own tablets, 8.14% own laptops, and 2.62% own other devices.

Table 1: Profile of Respondents

OTGHEU_2021_v8n8_637_t0001.png 이미지

5. Data Analysis and Results

 The relationships between the constructs were examined using Structural Equation Modeling (SEM). Statistical analysis was carried out using the SmartPLS 3 software. First, the validity and reliability of the measurement model were analyzed. Second, the structural model was analyzed to confirm the proposed relationships in the framework.

5.1. Measurement Model

5.1.1. Convergent Validity

 In analyzing the measurement model, the individual items’ reliability and the constructs’ composite reliability were evaluated. The individual items’ reliability was evaluated by looking at the significance of the individual items’ loadings. The loading of each individual item on its underlying construct should be ≥0.7, whereas the composite reliability (CR) and Cronbach’s Alpha (α) of each construct should be ≥0.7. Table 2 shows that all the item loadings on their theoretical constructs are ≥0.712. Additionally, the values of CR and α for each construct are ≥0.782.

Table 2: Convergent Validity

OTGHEU_2021_v8n8_637_t0002.png 이미지

 The validity procedure entails the assessment of the convergent and discriminant validity. Convergent validity was determined by looking at the average variance extracted (AVE) values for each construct (Hair et al., 2012). For the questionnaire to have convergent validity, the AVE value of each construct should be ≥0.5. Table 2 shows that all the AVE values are ≥0.615.

5.1.2. Discriminant Validity

 Discriminant validity must be ensured to confirm a good measurement model. Discriminant validity is established if a latent variable accounts for more variance in its associated indicator variables than it shares with other constructs in the same model. To satisfy this requirement, the Fornell-Lacker criterion was first assessed where each construct’s average variance extracted (AVE) must be compared with its squared correlations with other constructs in the model. According to Hair et al. (2013), the AVE values should be higher than the squared correlation of a construct and the other constructs in the model. Table 3 shows that all the conditions have been fulfilled.

Table 3: Discriminant validity-Fornell-Lacker Criterion

OTGHEU_2021_v8n8_637_t0003.png 이미지

Note: AT: Attitude, EE: Effort Expectancy, FC: Facilitating condition, PE: Perceived Ease of Use, PU: Perceived Usefulness, SI: Social Influence, UM: Use of M-Learning system.

 The second discriminant validity criterion is the Heterotrait-Monotrait ratio (HTMT) which demonstrates the average hetero-hetero-method correlation relative to the average mono-hetero-method correlation (Esposito et al., 2010). Table 4 indicates that the HTMT ratio of all the items and values are lower than the cut-off value of 0.90. Hence, the measurement model is confirmed to have met the discriminant validity criterion.

Table 4: Discriminant validity: Heterotrait-Monotrait Ratio (HTMT)

OTGHEU_2021_v8n8_637_t0004.png 이미지

Note: AT: Attitude, EE: Effort Expectancy, FC: Facilitating condition, PE: Perceived Ease of Use, PU: Perceived Usefulness, SI: Social Influence, UM: Use of M-Learning system.

5.2. Structural Model

 The next phase entails the evaluation of the proposed structural paths, specifically the explanatory power (R2) and path (regression) coefficients (β). Figure 2 shows that the six independent variables explained 72% (R2 = 0.720) of the variance in the dependent variable i.e. M-Learning system usage.

OTGHEU_2021_v8n8_637_f0002.png 이미지

Figure 2: Structural Model

5.2.1. Evaluation of the Structural Model

 Table 5 shows that effort expectancy (β = 0.242, P < 0.005), facilitating condition (β = 0.374, P < 0.005), perceived ease of use (β = 0.102, P < 0.005), and social influence (β = 0.11, P < 0.005) all have significant and positive effects on M-Learning system usage, and thus drive the usage of M-Learning. Meanwhile, attitude (β = 0.077, P > 0.005) and perceived usefulness (β = 0.0844, P > 0.005) have significant and negative effects on M-Learning, and thus hinder the usage of M-Learning.

Table 5: Results of the Structural Model Analysis (Hypotheses Testing)

OTGHEU_2021_v8n8_637_t0005.png 이미지

Note: **indicates significant at 1% level of significance based on t-statistics. AT: Attitude; EE: Effort Expectancy; FC: Facilitating condition; PE: Perceived Ease of Use; PU: Perceived Usefulness; SI: Social Influence; UM: Use of M-Learning system.

6. Findings and Discussion

 The findings strongly support Hypothesis 2 i.e. that effort expectancy positively affects M-Learning system usage. In short, M-Learning is effective and advantageous in attaining learning objectives and improving the students’ effort. This factor also improves M-Learning system usage intention. The greater the effort expectancy, the greater the M-Learning system usage. This finding is consistent with that of Samsudeen and Mohamed (2019) and Khan et al. (2020). The findings also support Hypothesis 3 which suggests that facilitating conditions have a positive and significant effect on M-Learning system usage. The items for facilitating conditions include the needed resources, relevant ICT infrastructures, requisite skills for accessing the course online, and the existence of a peer group for resolving M-Leaning inquiries. These factors positively affect M-Learning systems adoption. Ambarwati et al. (2020) and Saroia and Gao (2019) found similar results to that of the current study.

 Hypothesis 4 is supported and accepted. M-Learning usage requires very little effort with easy data access (factors that motivate people to use M-Learning). The greater the perceived ease of use, the higher the M-Learning system usage. Azizi and Khatony (2019) and Samsudeen et al. (2021) examined the effect of perceived ease of use on M-Learning and indicated similar findings to the current study. Hypothesis 6 is also supported, demonstrating that social influence significantly affects M-Learning systems usage as was proven by Azizi and Khatony (2019), Samsudeen and Mohamed (2019), and Jung (2018). Hypothesis 1 is not supported as the findings indicate that attitude strongly and negatively affects M-Learning system usage. Students’ negative attitude hinders their adoption of the M-Learning system, as was found by Azizi and Khatony (2019) and Saroia and Gao (2019). Hypothesis 5 is also rejected as the findings indicate that perceived usefulness has a negative effect on the usage of the M-Learning system. In short, M-Learning does not help in attaining learning objectives and improving the students’ efficiency.

7. Conclusion and Recommendation

 This study mainly explores factors that drive M-Learning system usage among higher education students. An empirical framework was developed based on a number of technology acceptance models. The findings indicate that the proposed model explains 72% of the variance in M-Learning usage for learning purposes. Effort expectancy, facilitating condition, perceived ease of use, and social influence were shown as key factors driving M-Learning usage. Meanwhile, attitude and perceived usefulness are indicated as the key barriers to M-Learning usage. Implications-wise, the current findings are beneficial for M-Learning providers and developers. The developers should design simple applications that fit the students’ needs. The applications must also be different than traditional learning styles and tools. The students must be able to identify the benefits of using M-Learning on their general learning performance.

 Limitations-wise, the study sample only focused on students from higher education institutions in the Batticaloa district. Future studies could incorporate students from other areas and examine the actual usage of M-Learning amongst university and college students. Finally, this study is quantitative in nature; future studies can employ a mixed method approach (qualitative and quantitative) to reach a broader understanding of M-Learning usage.


  1. Ambarwati, R., Harja, Y. D., & Thamrin, S. (2020). The role of facilitating conditions and user habits: A case of Indonesian online learning platform. The Journal of Asian Finance, Economics, and Business, 7(10), 481-489.
  2. Azizi, S. M., & Khatony, A. (2019). Investigating factors affecting medical sciences students' intention to adopt mobile learning. BMC Medical Education, 19(1), 1-10.
  3. Bartholomew, S. R., Reeve, E., Veon, R., Goodridge, W., Lee, V. R., & Nadelson, L. (2017). Relationships between access to mobile devices, student self-directed learning, and achievement. Journal of Technology Education, 29(1), 1-12.
  4. Cheon, J., Lee, S., Crooks, S. M., & Song, J. (2012). An investigation of mobile learning readiness in higher education based on the theory of planned behavior. Computers & Education, 59(3), 1054-1064.
  5. Davis, F. (1989). Perceived usefulness, perceived ease of use acceptance of information technology. MIS Quarterly, 13(3), 139-339.
  6. Esposito Vinzi, V., Chin, W. W., Henseler, J., & Wang, H. (2010). Handbook of partial least squares: Concepts, methods, and applications. New York: Springer.
  7. Gikas, J. G., & Grant, M. M. (2013). Mobile computing devices in higher education: Student perspectives on learning with cellphones, smartphones & social media. The Internet and Higher Education, 19(10), 18-26.
  8. Hair, F., Hult, G., & Ringle, M. (2013). A primer on partial least squares structural equation modeling (PLS-SEM). Thousand Oaks: Sage Publications.
  9. Hair, J., Sarstedt, M., & Ringle, C. (2012). An assessment of the use of partial least squares structural equation modeling in marketing research. Journal of the Academy of Marketing Science, 40(3), 414-433.
  10. Hamidi, H., & Jahanshaheefard, M. (2019). Essential factors for the application of education information system using mobile learning: A case study of students of the university of technology. Telematics and Informatics, 38, 207-224.
  11. Hamidi, H., & Chavoshi, A. (2018). Analysis of the essential factors for the adoption of mobile learning in higher education: A case study of students of the University of Technology. Telematics and Informatics, 35(4), 1053-1070.
  12. Herrington, A., & Herrington, J. (2007). Authentic mobile learning in higher education.
  13. Junco, R., & Mastrodicasa, J. (2007). Connecting to the net generation: What higher education professionals need to know about today's students. Washington, DC: NASPA.
  14. Jung, H. (2018). Trends and future directions of corporate e-learning contents. The Journal of Industrial Distribution & Business, 9(2), 65-72.
  15. Khan, U., Zhang, Y., & Salik, M. (2020). The impact of information technology on organizational performance: The mediating effect of organizational learning. The Journal of Asian Finance, Economics, and Business, 7(11), 987-998.
  16. Kukulska-Hulme, A. (2009). Will mobile learning change language learning? ReCALL, 21(02), 157-165.
  17. Malikhao, P., & Servaes, J. (2010). The media use of American Youngsters in the age of Narcissism: Surviving in a 24/7 media shock and awe - distracted by everything. Telematics and Informatics, 28(2), 66-76.
  18. Nawaz, S. S., & Mohamed, R. (2020). Acceptance of mobile learning by higher educational institutions in Sri Lanka: An UTAUT2 approach. Journal of Critical Reviews, 7(12), 1036-1049.
  19. Norris, D. M., Mason, J., & Lefrere, P. (2003). Transforming e-knowledge: A revolution in the sharing of knowledge. Ann Arbor, MI: Society for College and University Planning.
  20. Raza, S. A., Umer, A., Qazi, W., & Makhdoom, M. (2018). The effects of attitudinal, normative, and control beliefs on M-Learning adoption among the students of higher education in Pakistan. Journal of Educational Computing Research, 56(4), 563-588.
  21. Samsudeen, S. N., & Mohamed, R. (2019). University students' intention to use e-learning systems. Interactive Technology and Smart Education, 16(3), 219-238.
  22. Samsudeen, S. N., Thelijjagoda, S., & Sanjeetha, M. B. F. (2021). Social media adoption: Small and medium-sized enterprises' perspective in Sri Lanka. Journal of Asian Finance, Economics, and Business, 8(1), 759-766.
  23. Saroia, A. I., & Gao, S. (2019). Investigating university students' intention to use mobile learning management systems in Sweden. Innovations in Education and Teaching International, 56(5), 569-580.
  24. Sharma, S. K., Sarrab, M., & Al-Shihi, H. (2016). Development and validation of mobile learning acceptance measure. Interactive Learning Environments, 25(7), 847-858.
  25. Soetjipto, N., Kurniawan, G., Sulastri, S., & Riswanto, A. (2020). Supply chain analysis in public works: The role of work climate, supervision, and organizational learning. The Journal of Asian Finance, Economics, and Business, 7(12), 1065-1071.
  26. Tapscott, D. (1998). Growing up digital. New York: McGraw-Hill Companies.
  27. Venkatesh, V., Morris, M., Davies, G., & Davis, F. (2003). User acceptance of information technology: Toward a unified view. MIS Quarterly, 27(3), 425-478.
  28. Wang, D., & Lam, K. C. (2019). Relationship between ambidexterity learning and innovation performance: The moderating effect of redundant resources. The Journal of Asian Finance, Economics, and Business, 6(1), 205-215.
  29. Wang, T., Jung, C., Kang, M., & Chung, Y. (2014). Exploring determinants of adoption intentions towards enterprise 2.0 applications: An empirical study. Behaviour & Information Technology, 33(10), 1048-1064.