Development of the Korean Handwriting Assessment for Children Using Digital Image Processing

Abstract

The efficiency and accuracy of handwriting measurement could be improved by adopting digital image processing. This study developed a computer-based Korean Handwriting Assessment tool. Second graders participated in this study by performing writing tasks of consonants, vowels, words, and sentences. We extracted boundary parameters for each letter using digital image processing and calculated the variables of size, size coefficient of variation (CV), misalignment, inter-letter space, inter-word space, and ratio of inter-letter space to inter-word space. Children were also administered traditional handwriting and visuomotor tests. Digital variables from image processing were correlated with these previous tests. Using these correlations, we established a three-point scoring system that computed test scores for each variable. We analyzed inter-rater reliability between the computer rater and human rater and test-retest reliability between the first and second performances. The validity was examined by analyzing the relationship between the Korean Handwriting Assessment and previous handwriting and visuomotor tests. We suggested the Korean Handwriting Assessment to measure size, size consistency, misalignment, inter-letter space, inter-word space, and space ratio using digital image processing. This Korean Handwriting Assessment tool proved to have reliability and validity. It is expected to be useful for assessing children's handwriting.

1. Introduction

We communicate with others using language, which is usually spoken and written. The basis of accurate verbal communication is speech with clear pronunciation. Likewise, in order to communicate understandably through writing, it is essential to write legibly. Although typing can be used, handwriting is continuously emphasized because it may be used in a widevariety of situations. Therefore, handwriting legibility is needed for good writing. The mostessential components of handwriting legibility have been suggested to be form, size, alignment, and spacing [1]. These components can be measured with handwriting assessment tools including the Minnesota Handwriting Assessment [2] and Children’s Hand writing Evaluation Scale [3].

The inclusion of various elements in handwriting legibility means that handwriting requirescomplex skills. Let us suppose that a child writes a sentence from a book. The child looks at the letter (visual fixation), perceives the letter shape (visual perception), encodes the information of the sentence (memory), directs eyes to the notebook (eye movements), organizes how letters are positioned (planning), holds a pencil appropriately (fine motor control), remembers the letter (memory), and moves the pencil to produce the letter (eye-hand coordination). Perceptual-motor and cognitive functions are interwoven during handwriting [4, 5, 6]. In particular, visuomotor integration has been considered a necessary skill in the learning stage of writing [7, 8, 9]. Visuomotor integration as measured with the Beery-Buktenica Developmental Test of Visual-Motor Integration (Beery VMI) is positively correlated with hand writing legibility [7]. The ability to copy nine basic shapes (│, —, O, X, ＼, □, /, △)reflects children’s readiness to write [8, 9].

Writing is a fundamental skill for school-aged children to learn (Amundson, 1992). Forinstance, children write when taking notes and solving math problems. If a child’s writing ispoorly legible, the child would not receive a good score on what is written [10]. Children who have experienced frustration in writing do not want to engage in writing activities [11]. Since hand writing in childhood affects academic performance and socioemotional development, it isimportant to assess and intervene handwriting problems.

Standardized assessments to measure Korean writing in childhood include the hand writing subtest of the National Institute of Special Education-Basic Academic Competence Tests (NISE-B· ACT) [12] and the writing subscale of the Korean Version of the Learning Disability Evaluation Scale (K-LDES) [13]. Although the handwriting subtest of NISE-BACT(NISE-BACT-hand writing) directly measures children’s handwriting performance, it depends on the rater’s subjective decision. NISE-BACT-handwriting has an ambiguouscriterion for scoring one to three points. For example, one can score “3 points if the position and ratio of the letters are the same as those of the stimulus.” The writing subscale of K-LDES(K-LDES-writing) indirectly evaluates handwriting by the report of an adult who knows children & rsquo;s usual writing performance. These tools have the limitation of subjective evaluation as well as the advantage of quick assessment. Recently, Lim et al. [14] have developed anevaluation tool with objective criteria for school-aged children’s handwriting. The evaluationtool provides numerical criteria such as “0 points, if the longest letter is equal to or longer than 1.5 times the shortest letter.” However, a tester has to use a ruler and protractor. Thus, this tool requires time and effort. A tool that can assess Korean handwriting objectively in a short time is needed.

Computer-based handwriting assessment could satisfy the requirements of both objectivity and speed. In other countries, researchers have attempted to evaluate handwriting in computerenvironments. The Chinese Handwriting Assessment Program has been developed for a computer environment [15]. This image processing program can calculate the form, size, alignment, and spacing of a character from scanned writing. The processing result of the Chinese Handwriting Assessment Program was correlated with the score on the Tseng Hand writing Problem Checklist [15] and was found to be consistent across test-retest, indicating that the Chinese Handwriting Assessment Program is a valid and reliable instrument. The Minnesota Handwriting Assessment [2], a representative English hand writingassessment using subjective scoring, was implemented objectively in the tablet environment [16]. Results from the analysis of digitized data could differentiate between proficient and non-proficient handwriting.

In Korea, computer-based handwriting assessment for children has not been developed. Thus, the objective of the current study was to develop a Korean Handwriting Assessment tool based on computers using . We conducted this research in four stages: task development, collection of children’s writing data, establishment of the scoring system, and evaluation of its validity and reliability. The Korean Handwriting Assessment tooldeveloped in this study will enable us to measure Korean children’s handwriting quickly and objectively.

2. Method

2.1 Participants

Forty children in the second grade of elementary school (mean age, 8 years; age range, 7.07-8.10 years; 25 girls and 15 boys) participated in this study. Eighteen of them received the Korean Handwriting Assessment again two weeks after the main experiment to determinetest-retest reliability. Participants were recruited from two elementary schools. Writteninformed consent to participate in this study was obtained from each participant.

2.2 Procedure

This study had four phases: (1) developing writing tasks, (2) performing an experiment tocollect data, (3) establishing a scoring system, and (4) analyzing reliability and validity. During the experiment in the second phase, children performed not only the Korean Hand writing Assessment but also NISE-B·ACT-handwriting and Beery VMI. They wereadministered these tests individually for about 30 minutes in a quiet room of the school. The parent or teacher filled out a questionnaire of the K-LDES-writing about their child’shand writing problems.

2.3 Instruments

2.3.1 Korean Handwriting Assessment

We developed the Korean Handwriting Assessment based on elementary textbooks tomeasure handwriting performance with ecological validity. The Korean Hand writing Assessment consists of three subtasks: consonant-vowel, word, and sentence tasks. Thestimuli of the tasks were selected from first and second-grade textbooks for Korean and Korean language activities. The tasks are shown in Fig. 1.

Fig. 1. Tasks of the Korean Handwriting Assessment.

The consonant-vowel task included the 19 basic consonants (ㄱ-ㅎ) and 10 vowels (ㅏ, ㅑ,ㅓ, ㅕ, ㅗ, ㅛ, ㅜ, ㅠ, ㅡ, ㅣ). The stimulus for each consonant/vowel, with a size of 37.5 pt, was presented in a box 1.7 cm wide and 1.7 cm high. Another box for writing was placed 1.5cm below the stimulus box.

The word task included six types of character (아, 우, 와, 앙, 웅, 웡) as well as basicconsonants and vowels. These words were 동녘 ‘east’, 자유 ‘freedom’, 공원 ‘park’, 양말‘socks& rsquo;, 축하 ‘celebration’, 애벌레 ‘caterpillar’, 운동화 ‘running shoes’, 학용품 ‘school supplies & rsquo;, 미끄럼틀 ‘slide’, and 술래잡기 ‘hide-and-seek’. The sizes of the stimulus and writing box were the same as in the consonant-vowel task. The distance between the stimulus box and the writing box was 1.2 cm.

The sentence task included six types of characters. These sentences were 오랜만에 삼촌이우리 집에 오셔서 반가웠습니다 ‘I was glad of my uncle's visit to our home because it has been a while since he last came’ and 동생이 나무에 매달려 노는 걸 도와줘서 뿌듯해요 ‘Iam proud of helping my brother play while hanging in the tree.’ The stimulus sentences werepresented with a size of 30.8 pt. The writing box, 28.5 cm in width and 1.4 cm in height, was placed 0.5 cm below the stimulus.

We extracted the boundary parameters and calculated the size, size coefficient of variation (CV), and misalignment for each letter and the inter-letter and inter-word spaces. The values of these variables were averaged across letters or spaces in each task. Each variable value was then converted into three-point scores (0, 1, and 2). The consonant-vowel and word tasks haditem scores for size, size CV, and misalignment, while the sentence task also had item scores for inter-letter space, inter-word space, and the ratio of inter-letter space to inter-word space. The possible scores were 6 points for the consonant-vowel task, 6 points for the word task, and 12 points for the sentence task, for a possible total score of 24 points.

2.3.2 The handwriting subtest of the National Institute of Special
Education—Basic Academic Competence Test (NISE-B·ACT - handwriting)

The NISE-BACT aims to assess basic learning abilities in order to screen and diagnosestudents who need special education [12]. The NISE-BACT comprises reading, writing, and mathematics tests. We administrated the handwriting subtest of the writing test to assess the prerequisite skills for writing and letter form quality. The reliability of the NISE-BACTwriting subtest was confirmed by a Cronbach’s α of 0.91, the test-retest reliability of the hand writing subtest by correlation coefficients of 0.80-.0.90, and the validity of the hand writing subtest by a correlation coefficient of 0.48 with the writing subtest of the Korea Institute for Special Education—Basic Academic Achievement Tests (KISE-BAAT) [17].

2.3.3 The writing subscale of the Korean Version of the Learning Disability
Evaluation Scale (K-LDES-writing)

K-LDES is a parent/teacher-reported scale to screen children with learning disabilities and test treatment effectiveness [13]. K-LDES consists of listening, thinking, speaking, reading, writing, spelling, and math subscales. We used the handwriting problem of the writing subscale, for which a higher score indicates a greater severity of the problem. The reliability of K-LDES was shown by a Cronbach’s α of 0.92-0.97 and the validity of the writing subscale by correlation coefficients of 0.44-0.48 with the Wechsler Intelligence Scale for Children, Fourth Edition (KEDI-WISC) [18] and 0.40 with the Basic Learning Test (BLT) [19].

2.3.4 Beery-Buktenica Developmental Test of Visual-Motor Integration
(Beery VMI)

The Beery VMI measures the ability to integrate vision and motor skills by copyinggeometric shapes [20]. The test consists of a visual-motor integration task (VMI), visual perception supplemental task (VP), and motor coordination supplemental task (MC). Thereliability of Beery VMI is shown by a Cronbach’s α of 0.92-0.94 and interrater correlation coefficient of 0.79-0.90, and the validity of the test by correlation coefficients of 0.62-0.80 with the Bender Visual-Motor Gestalt Ⅱ (BGT-Ⅱ) [21] 0.33-0.59 with the visual motion integration test of the Korean Developmental Test of Visual Perception (K-DTVP-2) [22], and 0.39-0.74 with KEDI-WISC [18].

2.4 Data analysis

2.4.1 Computation of variables

In order to compute the variables of the Korean Handwriting Assessment, we extracted the boundaries of the top, bottom, left, and right of each letter. Fig. 2 shows the parameters for the variables. We calculated the variable values for each task using these parameters. Values of each variable for the first test were derived by the computer rater using digital image processing and the human rater using a ruler. Values for the second test (retest) were measured by the computer rater.

Fig. 2. Digital parameters of the Korean Handwriting Assessment.

\(\text { Size }=\frac{1}{n} \sum_{i=1}^{n}\left(H_{i} \times w_{i}\right)\)       (1)

\(\text { Size } C V=\frac{\sqrt{\frac{1}{n-1} \sum_{i=1}^{n}\left[\left(H_{i} \times W_{i}\right)-\overline{(H \times W)}\right]^{2}}}{\frac{1}{n} \sum_{i=1}^{n}\left(H_{i} \times W_{i}\right)}\)       (2)

\(\text {Misalignment}=\frac{1}{n} \sum_{i=1}^{n}\left|y_{i}\right|\)       (3)

\(\text { Inter - letter space }=\frac{1}{27} \sum_{i=1}^{27} d l_{i}\)       (4)

\(\text { Inter - word space }=\frac{1}{11} \sum_{i=1}^{11} d w_{i}\)       (5)

\(\text {Space ratio }=\frac{\text {Inter-letter space}}{\text {Inter-word space}}\)       (6)

 

2.4.2 Scoring

The value of each variable was converted to an item score for each task. We scored hand writing performance with a three-point scale system (0, 1, and 2). The scoring system wasestablished based on the digital variable property inferred from the correlation between the digital variable and the raw score on the previous test. The size score decreased as the size of the letter decreased. We scored 2 if the size was greater than 2/3 of the stimulus size, 1 if thesize was greater than 1/2 of the stimulus size, and 0 if the size was less than or equal to 1/2 of the stimulus size. The size CV score decreased as the size inconsistency increased. We scored 2 if the size CV was less than 0.2, 1 if the size CV was less than 0.4, and 0 if the size CV was greater than or equal to 0.4. The misalignment score decreased as letters were placed furtheraway from the horizontal midline. We scored 2 if the misalignment was less than 0.1 cm, 1 if the misalignment was less than 0.2 cm, and 0 if the misalignment was greater than or equal to 0.2 cm. The inter-letter space score decreased as the distance between letters increased. Wescored 2 if the inter-letter space was less than 0.2 cm, 1 if the inter-letter space was less than 0.5 cm, and 0 if the inter-letter space was greater than or equal to 0.5 cm. The inter-word spacescore decreased as the distance between words decreased. We scored 2 if the inter-word space was greater than 0.8 cm, 1 if the inter-word space was greater than 0.6 cm, and 0 if the inter-word space was less than or equal to 0.6 cm. The space ratio score decreased as the increase of the distance between letters was relatively larger than the increase in the distance between words. We scored 2 if the ratio was less than 33%, 1 if the ratio was less than 50%, and 0 if the ratio was greater than or equal to 50%.

2.4.3 Analysis of reliability and validity

Interclass correlation coefficients (ICCs) were computed to evaluate inter-rater and test-retest reliability. Pearson correlation analysis was conducted to validate total and subtests cores on the Korean Handwriting Assessment. Pearson correlation coefficients were alsoused to evaluate the relationship between the digital variables and the results of previous tests. We performed data analysis with Windows SPSS 23.0. The level of statistical significance was set at α = .05 .

3. Results

3.1 Participant characteristics

Results of previous assessments of children participating in the study are shown in Table 1.

Table 1. Results of previous assessments

3.2. Properties of the variables

Values of the variables of the Korean Handwriting Assessment are shown in Table 2.

Table 2. Values of variables of the Korean Handwriting Assessment

To understand the properties of the variables, we analyzed the correlation coefficients between variable values and previous assessment scores. The size was positively correlated with VP (consonant-vowel task, r = .382, p < .05; word task, r = .373, p < .05; sentence task, r= .314, p < .05). The size of the word task was negatively correlated with K-LDES-writing (r =−.359, p < .05). These results related to the size indicated that the larger the letter size, the better the visual perception and the fewer the writing problems. The size CV was positively correlated with K-LDES-writing (word task, r = .322, p < .05; sentence task, r = .471, p < .01). The size CV of the word task was negatively correlated with NISE-B·ACT-handwriting (r =−.400, p < .05). The results related to the size CV indicate that the smaller the size CV, the better the handwriting performance and the fewer the writing problems.

The misalignment was negatively correlated with VMI (consonant-vowel task, r = −.339, p< .05; word task, r = −.329, p < .05), VP (consonant-vowel task, r = −.338, p < .05), MC (word task, r = −.313, p < .05), and NISE-B·ACT-handwriting (r = −.425, p < .01). Themisalignment was positively correlated with K-LDES-writing (word task, r = .360, p < .05; sentence task, r = .374, p < .05). These results of misalignment indicate that the smaller themisalignment, the better the visuomotor integration and handwriting performance and the fewer the writing problems.

The inter-letter space showed negative correlations with MC (r = −.420, p < .01) and NISE-B· ACT-hand writing (r = −.322, p < .05) and a positive correlation with K-LDES-writing (r = .549, p < .01), indicating that the smaller the inter-letter space, the better the motor coordination and handwriting performance and the fewer the writing problems. The inter-word space was positively correlated with NISE-B·ACT-handwriting (r = .441, p < .01), indicating that the larger the inter-word space, the better the handwriting performance. Thespace ratio was negatively correlated with MC (r = −.466, p < .01) and NISE-B· ACT-hand writing (r = −.466, p < .01), but positively correlated with K-LDES-writing (r = .518, p < .01), indicating that the smaller the space ratio, the better the motor coordination and handwriting performance and the fewer the handwriting problems.

3.3 Validity and reliability of Korean Handwriting Assessment

Score results of the Korean Handwriting Assessment are shown in Table 3.

Table 3. Scores of the Korean Handwriting Assessment

Table 4 shows the inter-rater reliability between the computer rater and the human rater and test-retest reliability between the first and second tests. The Korean Handwriting Assessment proved to have excellent inter-rater and test-retest reliabilities.

Table 4. Reliability of the Korean Handwriting Assessment

Table 5 shows concurrent validity between the Korean Handwriting Assessment and previous assessments including NISE-B·ACT-handwriting, K-LDES-writing, VMI, VP, and MC. The Korean Handwriting Assessment was found to be valid, showing correlations with hand writing performance/problem and visuomotor integration.

Table 5. The Validity of the Korean Handwriting Assessment

4. Discussion

This study described the development of the computer-based Korean Hand writing Assessment. First, we made tasks for the Assessment concerning consonants, vowels, words, and sentences from elementary textbooks. Second, we collected writing data from second-grade elementary school children. We administered the Korean Handwriting Assessment with NISE-B· ACT-hand writing, K-LDES-writing, and Beery VMI to evaluate the concurrent validity. Two weeks later, we tested about half of the children again with the Korean Hand writing Assessment to examine test-retest reliability. Third, we computed values of therespective variables, including size, size CV, misalignment, inter-letter space, inter-wordspace, and space ratio using digital imaging processing. We then established a 3-point scoring system using the relationship between variables and the results of the NISE-B· ACT-hand writing, K-LDES-writing, and Beery VMI tests. Fourth, we analyzed the validity and reliability of the Korean Handwriting Assessment. Concurrent validity analysesindicated that the Korean Handwriting Assessment could measure handwriting ability. The results of test-retest and inter-rater reliability analyses suggest that the Korean Hand writing Assessment can measure handwriting consistently across time and raters.

Through this study, we have provided an objective and rapid computerized assessment toolto test Korean children’s handwriting. Traditional tools to assess Korean hand writing performance for children depend on quick subjective ratings with arbitrary criteria [12] orlaborious objective ratings using a ruler [14]. There has been a trade-off between speed and objectivity in these traditional tools. However, the Korean Handwriting Assessment cancompute quantitative values accurately and rapidly.

The Korean Handwriting Assessment is a unique computerized handwriting assessment tool for Korean. In other languages, there have been several handwriting assessment tools incomputer environments. For Chinese, the written paper can be transformed into a digitalimage from which the assessment program can compute the variables of size, intercharacterdistance, distance from the midline, and similarity between the written character and the model character [15]. Each variable of the Chinese Handwriting Assessment Program was correlated with the corresponding item of the previous Tseng Handwriting Problem Checklist[23]. For English, the traditional handwriting assessment, the Minnesota Hand writing Assessment [2], was implemented on a tablet [16]. The quantitative value of variables from thetablet was comparable with subjective scores obtained manually. These studies have contributed to the development of computer-based handwriting assessment, suggesting the great promise and utility of a computerized tool. The current study provides additionalevidence for the usefulness of handwriting digital variables processed by computers. Furthermore, we attempted to produce a scoring system derived from experimental data. Thescores obtained according to our criteria provide an interpretable summary of performance. For example, if the value of size is 0.5, we could not infer the meaning of the number. However, if the score on size is 2 out of 0, 1, and 2, we may interpret this as indicating that the child wrote letters with an appropriate size. Thus, the Korean Handwriting Assessment could be used as an interpretable computerized assessment tool.

The Korean Handwriting Assessment measures the variables of size, size CV, misalignment, inter-letter space, inter-word space, and space ratio. Because we chose digital variables that could be calculated from the boundary parameter, we did not measure form variables of hand writing, such as letter reversals or stroke omission. Although the Korean Hand writing Assessment does not include all aspects of handwriting, it measures essential components oflegibility. In particular, size and space have been suggested to be useful variables todifferentiate between proficient and non-proficient handwriting [2].

Properties of good handwriting were found to be large and constant size, alignment in the center, small inter-letter space, and large inter-word space. This is consistent with previous findings [24] except for the results for the size variable. Our results showed that letter size was positively correlated with visual perception and negatively correlated with hand writing problems, which differs from previous findings that children with writing disabilities wrotelarger characters than children without such disabilities [24]. This discrepancy could beattributed to a difference in tasks. The current study used a near-point copying task whereasthe study by Shin and Cho [24] did not present visual stimuli because they measured phonological error. In the current study, children with good visual perception might have tried to write letters in a size similar to that of the stimulus letters. In future research, it is necessary to systematically study how children’s letter size varies according to the task.

Until now, there has been no computer-based handwriting assessment for Korean children. It is thus difficult to compare the approach proposed in this study with other computer-based methods. However, our approach could be partially compared with the method of Lim et al. [14]. Size consistency is a representative item shared across studies. To score size consistency in Lim et al.’s study [14], the rater measured the difference between the largest and the smallest letters. When the difference was less than 1.5 times, 1 point was given, otherwise 0 points were given. We analyzed the size consistency of handwriting of the current study with the scoring system of Lim et al. [14]. All children except one were given 0 point, and the average score was 0.02. This is comparable to Lim et al.’s results [14], which reported almost 0 points for size consistency in second-grade elementary school children. The scoring system for size consistency in Lim et al. [14] has the limitations that only some letters were included in the analysis and the criterion was so strict that a floor effect occurred. However, the present study proposed a stepwise criterion that could differentiate handwriting performances. We computed the size CV for all letters and assigned a score of 2 if the CV was less than 20%, 1 if the CV was less than 40%, and 0 if the CV was greater than or equal to 40%. The average CV and the mean score on the sentence task were 33% and 0.83, respectively. The current study has implications for providing systematic scoring criteria for handwriting performance on allletters.

The validity of the Korean Handwriting Assessment was supported by its strong correlation with the NISE-BACT-handwriting and K-LDES-writing tests and medium correlation with the Beery VMI. The level of correlation was comparable to that between the NISE-BACT-hand writing and K-LDES-writing (r = −.684, p < .001) from the data of the current study. The reliability was excellent, with correlation coefficients over .9 between ratersor between test and retest. Given that the NISE-BACT-handwriting test has a test-retestreliability of .768 and the K-LDES-writing test has Cronbach’s α of .94, the reliability of the Korean Handwriting Assessment is thus also promising. Taken together, the Korean Hand writing Assessment is a valid and reliable measurement tool that provides Koreanchildren & rsquo;s handwriting legibility.

This study focused on second-grade elementary school children without writing disabilities. Hand writing ability is affected by age and disorders such as developmental coordinationdisorder [25, 26], attention deficit hyperactivity disorder [27], and learning disorders [28]. In the future, we need to examine a large sample of children in various age groups and establishnorms to interpret individual data by comparing them with their peers. Also, future researchinvolving clinical groups would allow us to understand how handwriting is related todysfunctions and disorders.

5. Conclusion

This study describes the developmental procedure of the computer-based Korean Hand writing Assessment through digital imaging. The Korean Handwriting Assessment was found to have sufficient validity and reliability to measure handwriting performanceadequately. This is the first study to assess the handwriting ability of Korean children viacomputers objectively and speedily. The Korean Handwriting Assessment is expected to beexpanded into tele-assessment via the Internet.