Vidyanusa Mathematic Learning Systems Based on Digital Game by Balanced Design Approach

Abstract

Educational games offer an opportunity to engage and inspire students to take an interest in every subject material in school. The "fun" obtain when playing games become a trigger for the use of games in learning. However, there are doubts whether the players actually learn while they are having fun. Vidyanusa is an Online Mathematics Education Game being developed by Crayonpedia Education Ecosystem in Indonesia. The learning goal of Vidyanusa is to engage junior high school students in learning mathematics. In this paper, we design the Vidyanusa game material Functions and Relations by using Balanced Design Approach. This approach has three models in succession; the Content Model outlines the purpose of the game, the Task Model maps out the mission, and the Evidence Model outlines student measurement. This paper will then discusses the quality of games produced in term of Usability factor for effective results and objective. The measurement of the game was carried out based on International Standard ISO/IEC 9126-1 FDIS about Software Quality Product.

1. INTRODUCTION

An international agency Organization Economic Cooperation and Development (OECD) conducts a survey through a Programme of International Student Assessment (PISA) on the educational system and the ability of the students in the school. This survey is carried on every three years and involved 65 countries representing 80% of the global economy of the world. Test participants are students aged 15-16 years. Based on the survey results, in 2013 Indonesia ranked 64 out of 65 countries. In a survey earlier in 2009, Indonesia is ranked 59. The purpose of this survey is to measure student learning outcomes in Mathematics, reading, and Science [1-2]. Based on the survey, Mathematic lesson aims to measure understanding of identification as well as the use of the mathematical foundations needed by someone in daily life [1-2]. In Indonesia, the characteristics of the basic objects that are studied in mathematics are abstract covering: facts, concepts, or operating rules and principles. At the moment, often students’ in Math lesson act passively by just paying attention to the teacher who writes formulas on the board. This is one of the causes why the student's understanding of mathematics and lead students dislike math lesson because it is considered difficult. As a solution for creating a fun learning, Game Based Learning approach can be use as one of the alternatives.

Game Based Learning (GBL) is a method of learning that use digital games as a medium to reach the learning objectives. In GBL, students are actively doing experiences, understanding the game, and resolving the problems that exist in it [5]. Normally, students will feel good when they can solve mysteries in the game. The "fun" obtained when playing games become a trigger for the use of games in learning. However, frequently there are doubts whether the players actually learn while they are playing game? This becomes important when teachers conduct the assessment. For the purpose of learning, students' understanding towards the meaning of the game is very important.

In this paper, we provide a solution by making a game design learning. This design should be able to answer questions about what knowledge wants to achieve, what tasks the student will do in the game, and how the measurements is performed [5]. Harmonization of the three answers of these questions will produce a balanced game design and is expected to make learning not only fun but also more meaningful.

 

2. RELATED WORKS

2.1 The Lure of the Labyrinth

The Lure of the Labyrinth is a game developed by The Education Arcade at the MIT. The target audience of this game is middle school students, and its primary goal is the enhancement of pre-algebra math learning, with a secondary goal of improving literacy. It is a long-form puzzle adventure game played over many sessions, with a persistent narrative that evolves over time. In order to complete the game players must navigate complex mathematical spaces, and solve puzzles that embody the big ideas of mathematics [10]. The mission of this game is to save a pet who was abducted to another place. An atmosphere full of mystery and darkness can be found when playing this game. That impression was further felt with the selection of characters in the form of various forms of monsters and a thrilling sound.

2.2 Radix Endeavor

The Radix Endeavor is a Massively Multi-player Online Game (MMOG) being developed by The Education Arcade at the Massachusetts Institute of Technology, designed to improve learning and interest in STEM in high school students. The content specifically focuses on statistics, algebra, geometry, ecology, evolution, genetics, and human body systems. Players take on the role of mathematicians and scientists and embark on quests that encourage them to explore and interact with the virtual world through math and science. Players become embedded in a narrative in the world, where they encounter a villain who does not believe in the practices of science. Players have to reason about science issues applicable to game characters' daily lives, refute the unscientific claims of the villain, and make choices based on what they consider to be valid evidence. The goal is to have a culminating quest activity at the end of each quest line where students will apply what they have learned to a problem or new situation [11].

2.3 Comparison

This is a brief comparison between the proposed system and previous system on education game. The comparison result between our system and the related work display in Table 1 below.

Table 1.Comparison the system with related works

 

3. LITERATURE REVIEW

3.1 Balanced Design

Balanced Design is an approach to design educational games. The idea was created by a team from the MIT Scheller Teacher Education Program and Education Arcade staff in 2014. The purpose of this approach was made not to impose a new game development method, but rather to present a learning design approach to be integrated with current best practices in game design [5]. Balanced design was adapted from the fundamental structure of Evidence-Centered Design Model, an approach to constructing educational assessments in terms of evidentiary arguments. In Evidence Centered Design, there are six steps to review whether or not the students has achieved the standard, namely: define the domain, define claims to be made, define assessment targets, define evidence required, develop Task Model, and develop items or performance task [6]. Balanced design is a simpler approach that focuses on instructional design approach to integrate best practices in game design. This approach emphasizes on proportionality between the content of games with game design [5].

In Balanced Design elements of playful engagement and fun as it is on GBL remains the primary concern in designing games. The advantages of this approach are a concern for the purpose of topic subjects that made the game. This aims to create a better and more effective game for students. Using a Balance Design creates an educational game where the learning goals, game mechanics, and assessment about the students play and performance are aligned. To achieve the alignment, there are three models that must be met, namely Content Model, Task Model, and Evidence Model. For each model, there are several key questions as a reference for the achievement of the objectives to be achieved. All three methods have a relationship and function respectively as illustrated in above Fig. 1 [5].

Fig. 1.Elements of Balanced Design.

3.1.1 Content Model

Content models used to describe any knowledge, skills and abilities to be achieved by students. In each subject there is a syllabus about what material will be taught. The teacher should make a lesson plan based on this syllabus. Lesson plan outlines the core competencies, an indicator of a student’s success, and the elaboration of learning activities. A content model can be likened as a lesson plan in a shorter version.

3.1.2 Task Model

The task model aims to describe the main tasks in the game and how the situation in the game is made to get the kind of evidence required. This model explains the Mission of what the player will do as well as description step by step to do the players. It also explained about the game features, tasks, and experience the students will engage in.

3.1.3 Evidence Model

Evidence is related to measurement. At this stage of discussion about how the measurements to be performed and the behavior and performance of what it will be used. The evidence model defines the rules for how to interpret a learner’s performance on each task and how the content model should be updated given this interpretation [5].

3.2 Curriculum 2013

Curriculum 2013 (K-13) is a curriculum that is implemented by the Ministry of Education and Culture of the Republic of Indonesia to replace Curriculum Education Unit which has been in force for 6 years. This curriculum has three aspects of assessment, namely: Cognitive, Affective, and Psychomotor [12].

3.2.1 Cognitive Domain

In 1956, Benjamin Bloom wrote "Taxonomy of Educational Objectives: Cognitive Domain", and since then the description of the six-level description of the thought process been widely adapted and used in different contexts. List of cognitive processes, compiled and sorted from the most simple, given back the knowledge that has been owned, to the most elaborate, that decided the value and benefits of an idea.

Cognitive domain was the realm of discussion related to understanding and knowledge. Assessment of cognitive aspects indicator intended to measure the achievement of learning outcomes in terms of intellect, the ability to dig and process information or knowledge. Cognitive assessment consists of six stages, called Bloom’s Taxonomy, where this theory has previously been revised by Anderson, L. W and Krathwohl in her book "A taxonomy for learning, teaching and assessing: A revision of Bloom’s Taxonomy of educational objectives" [13]. Six stages namely: remembering, understanding, applying, analyzing, evaluating, and creating.

3.2.2 Affective Domain

Affective learning is demonstrated by behaviors indicating attitudes of awareness, interest, attention, concern, and responsibility, ability to listen and respond in interactions with others, and ability to demonstrate those attitudinal characteristics or values which are appropriate to the test situation and the field of study. Similar with cognitive objectives, affective objectives can also be divided into a hierarchy (according to Krathwohl). This area is concerned with feelings or emotions. Again, the taxonomy is arranged from simpler feelings to those that are more complex [14]. The aspects are: receiving, responding, valuing, organization, and characterization by value set.

3.2.3 Psychomotor Domain

The psychomotor domain deals with manual or physical skills. It is the "doing" domain. The table below outlines the five levels in this domain and verbs that can be used to write learning objectives [15].The aspects are: imitation, manipulation, precision, articulation, and naturalization.

3.3 Software Quality Product

According to the rules McCall, a way of measuring the quality attributes hierarchically structured, where the top level (high-level attribute) is called factoring (factor), and the lower level (low level attribute) is called the criteria. Factor indicates attributes product quality from the perspective of the user. While the criteria are product quality parameters from the perspective of its own software. Factor and these criteria have a causal relationship (cause-effect) [16]. The formula is:

while, Fa : total value of factor a, w: weight of the criteria, and c: value of the criteria.

From equation (1), there are several stages in the calculation as followings:

Stage 1: determine the criteria used,

Stage 2: specify the weight (w) of each of the criteria (usually, 0 ≤ w ≤ 1),

Stage 3: specify the scale of the value of each criteria (i.e. 0 ≤ w ≤ 1),

Stage 4: give value to each criteria, and

Stage 5: calculate the total value of Fa [16].

Based on ISO/IEC FDIS 9126-1 Standard, software quality attributes categorized into six characteristics, namely: functionality, reliability, usability, efficiency, maintainability and portability [17]. Each characteristic divided into sub characteristics. Usability factor is the capability of the software to be understood, learned, used, and attractive to user, when used under specified conditions. The criteria from usability factor are:

• Understandability: the capability of the software product to enable the user to understand whether the software is suitable, and how it can be particular tasks and conditions of use.

• Learnability: the capability of the software product to enable the user learn its application.

• Operability: the capacity of the software product to enable the user to operate and control it.

• Attractiveness: the capability of the software to be attractive to the user.

• Usability compliance: the capability of the software product to adhere to standards, conventions, style guides or regulations relating to usability.

 

4. SYSTEM ANALYSIS

4.1 Analysis of Mathematics Subject Using Balance Design

Totally, there are 77 core competencies in the Subjects of mathematics junior level classes 7, 8 and 9. This competency includes aspects of affective, cognitive, and psychomotor. Vidyanusa (Vidya: Knowledge, Nusa: Island) summarizes the aspects of cognition and transforming them into 23 missions. These thesis will explain two mission, namely mission 21 and mission 22. Table 2 below is design of Vidyanusa System based on Balance Design on that missions.

Table 2.Design of Vidyanusa System Mission 21 & 22

4.2 Assessment Analysis

There are three domains of assessment of student learning outcomes, namely cognitive, affective and psychomotor. The third domain has a provision each assessment. The following are the explanation.

4.2.1 Assessment of Cognitive Domain

In Cognitive domain, assessment of the student’s understanding viewed from two aspects, namely Score and Play Point. To get the results of the assessment, there are assumptions that must be applied, which is: every game has indicators of achievement by different amounts, each indicator is assumed to be one scene, and every scene has a value of 100 points if players wins the game. The following is how to calculate the value of student understanding.

a. Score: The score is obtained from the value of the average score of the students play each one scene at one time class meeting. Score formula:

b. Play Point: The Play Point is the number of times to play the game during the first meetings:

3 times = 100 points, 2 times = 50 points, 1 time = 10 points, and 0 times = 0 points.

From the two calculation above, we can get Total Point for Cognitive domain with formula (3):

where TP: Total Point, S: Score, PP: Play Point, and i: the sum of indicators.

Description of achievement of student competency can be classified as follows:

• 76-100: Students are very capable to resolve the real problems using relations, stating the relationship between two objects, and resolve the real problems using functions method.

• 51-75: Students are able to understand all the indicators of achievement.

• 26-50: Students are quite able to understand 50% indicators of achievement.

• 0-25: Students are not able to understand the indicators of achievement.

4.2.2 Assessment of Affective Domain

Based on the Curriculum of 2013 state of Indonesia, affective assessment related to attitude like spiritual, and social attitude. Social attitude divine into five category, namely: responsibility, discipline and consistent, confident and initiative, honesty and self-reliance, and mutual cooperation and compassion. Every aspect has their own calculation. Table 3 is the explanation of assessment of affective domain in social attitude.

Table 3.Assessment of Social Attitude

4.2.3 Assessment of Psychomotor Domain

The psychomotor domain deals with manual or physical skills. It is the "doing" domain. Assessment for psychomotor domain divide into three groups as shown in Table 4.

Table 4.Assessment of Psychomotor Domains

 

5. SYSTEM IMPLEMENTATION AND EXPERIMENT RESULT

5.1 Mission 21 (Title: Build the dam, Material: Relations, Location: Dam)

Fig. 2.Opening Mission 21.

Fig. 3.Instruction.

Fig. 4.Winning Condition.

Fig. 5.Failure Condition.

5.2 Mission 22 (Title: Distribute crystal stone to build the dam, Material: Functions, Location: Dam)

Fig. 6.Opening Mission 22.

Fig. 7.Instruction.

Fig. 8.Winning Condition.

Fig. 9.Failure Condition.

5.3 Software Product Quality Measurement

In the assessment of the quality of products, the number of respondents used is 20 people came from Indonesian college students in Korea with different background study. The selection of respondents is done by random sampling. It is intended that the assessment carried out completely objective[18]. Respondents asked to play the game Vidyanusa missions 21 and 22, then given a questionnaire in the form of questions about the usability of the product. To measure software product quality, there are several stages as followings:

Specify the weight (w) of each of the criteria (usually, 0 ≤ w ≤ 1). We assume the weight of every criteria as table below:

Table 5.Weight of criteria

Specify the scale of the value of each criteria (i.e. 0 ≤ w ≤ 1) and give value to each criteria.

Table 6.Scale and value of each criteria

Calculate the total value of Fa = (ω1c1 + ω2c2 + ⋯ + ωncn), in equation (1), while, Fa: total value of factor a, w: weight of the criteria, and c: value of the criteria. From the Table 5.2 we can calculate Fa for every mission as follow:

Mission 21: Fa= 0.2(7)+0.3(8)+0.2(9)+0.2(7)+0.1(6)=1.4+2.4+1.8+1.4+0.6=7.6

Mission 22: Fa= 0.2(9)+0.3(8)+0.2(7)+0.2(8)+0.1(7)= 1.8+2.4+1.4+1.6+0.7=7.9

From the above calculations, the total value of the results obtained in terms of usability mission 21 is mission 22 is 7.6 and 7.9. Referring to Table 7, the total value derived by the two missions is included in the category of high quality. This indicates, the mission of 21 and 22 have a high quality of in terms of usability.

Table 7.Metrics achievement of Total Value

 

6. CONCLUSION AND FURTHER STUDY

We analyzed Mathematic Subject in Educational Game Based Learning using Balanced Design Approach. By using a Balanced Design approach, it enables learning goals, game mechanics, and assessments in educational games aligned with student performance. Usability factor is the capability of the software to be understood, learned, used, and attractive to user, when used under specified conditions. According to the experiment, the results of Software Quality Product forevery mission are respectively 7.6 and 7.9. The total value obtained by the mission indicates that 21 missions and 22 missions possess a high quality in terms of usability. To be able to measure the implementation of Balanced Design approach more thoroughly, we are planning to do some research to the students of Junior High School in Indonesia. This study aims to measure evidence model of Balanced Design. The results obtained by the students after playing the game can be used as input for the assessment of cognitive aspects. While attitudes and student activity will be the input for affective and psychomotor aspects. For future works we will also more focus on the design interface side. Vidyanusa game is expected to be more interesting and responsive, for example by adding a game with different difficulty levels.