• Journal of Gifted/Talented Education
• /
• v.25 no.1
• /
• pp.119-138
• /
• 2015

This study analyzed the academic achievements on above-level testing of mathematics, physics, chemistry, and English in newly entering students of science specialized high schools. It can be expected that newly students of science high specialized schools have reached ceiling level in the middle school mathematics and science academic scores. Above-level testing(or off-level testing) is a test tool used to evaluate student's ability which are above-grade level. In this study, above-level testing tools were used to develop the same type examination paper of the 2013 Korean College Scholastic Ability Test(CSAT) in mathematics, physics, chemistry, and English. The conclusions of this study were as follow: First, the academic achievement level of science specialized high school freshmen were higher the average level of general high school senior because that over 50% of them are within the 5 grade of CSAT in mathematics, physics, and chemistry. In English, 19.3% science specialized high school freshmen have reached within the 5 grade of CSAT. Second, as a result of examining characteristics of academic achievement with respect to units of subjects, in mathematics, it was showed that the academic achievement of 'continuity and limit of a function' unit was higher, 'statistics' unit was lower. In physics, the academic achievement of 'Electricity and Magnetism' unit was higher, 'Waves and particles' unit was lower. In chemistry, the academic achievement of 'compounds in life' unit was higher, 'Air' unit was lower. In English, the academic achievement of 'practical sentence' of reading area was higher, 'Sentence' of writing area was lower. In conclusion, above-level testing provided a good strategy for identifying and determining appropriate programming interventions for gifted students who are two or more grade levels above their age-mates in achievements, aptitude, or ability.

• English Language & Literature Teaching
• /
• v.16 no.4
• /
• pp.153-184
• /
• 2010

On the bases of the 5 principles and the 4 criteria for designing and developing of the listening and the reading test of National English Ability Test (NEAT), this study presents Item-Types Decision-Making Model as a blueprint for designing and constructing the two tests. It sets up the criteria for validating item types, designs a modular type of test specifications, constructs an item-types bank, and specifies a complementary type of test specifications of the two tests. To gather all these threads up, it constructs Item-Types Decision-Making Model which consists of such components as the item-type pool, the validity criteria and the procedures of testing item types, the item-types bank, the modular and the complementary type test specification. Thus, it shows how the Model works in developing and constructing the two level-differentiated listening and reading tests (the 2nd and the 3rd rank) of NEAT. Finally, it discusses some implications and applications of the Model to the two level-differentiated tests (the A and the B type) of 2014 CSAT (College Scholastic Ability Test) systems, National Assessment of Educational Achievement (NAEA), and classroom testing. In conclusion, Item-Types Decision-Making Model functions as a testing template in an item development system and as a matrix in an item-types bank system.

• KIPS Transactions on Software and Data Engineering
• /
• v.4 no.11
• /
• pp.509-520
• /
• 2015

College Scholastic Ability Test(CSAT) is a primary test to evaluate the study achievement of high-school students and used by most universities for admission decision in South Korea. Because its level of difficulty is a significant issue to both students and universities, the government makes a huge effort to have a consistent difficulty level every year. However, the actual levels of difficulty have significantly fluctuated, which causes many problems with university admission. In this paper, we build two types of data-driven prediction models to predict correct answer rate and to identify significant factors for CSAT English test through accumulated test data of CSAT, unlike traditional methods depending on experts' judgments. Initially, we derive candidate question-specific factors that can influence the correct answer rate, such as the position, EBS-relation, readability, from the annual CSAT practices and CSAT for 10 years. In addition, we drive context-specific factors by employing topic modeling which identify the underlying topics over the text. Then, the correct answer rate is predicted by multiple linear regression and level of difficulty is predicted by classification tree. The experimental results show that 90% of accuracy can be achieved by the level of difficulty (difficult/easy) classification model, whereas the error rate for correct answer rate is below 16%. Points and problem category are found to be critical to predict the correct answer rate. In addition, the correct answer rate is also influenced by some of the topics discovered by topic modeling. Based on our study, it will be possible to predict the range of expected correct answer rate for both question-level and entire test-level, which will help CSAT examiners to control the level of difficulties.

• Journal of The Korean Association For Science Education
• /
• v.31 no.8
• /
• pp.1158-1174
• /
• 2011

In this study, we examined the compositions, basic principles, and the area of the National Higher Education Entrance Examination (GaoKao) in 2009, we also analyzed the categories and characteristics of items. Also, the GaoKao was analyzed in terms of test specifications, the number of items, item patterns, difficulty levels, and implications of the College Scholastic Ability Test(CSAT) were explored. Results show Natural Science section of the National Test 1, 2 are 300 points per 150 min, and Natural Science, and Chemistry of Shanghai is 150 points each per 120 min. Also, the GaoKao contained multiple choice and fill in the blanks questions, and the description items are composed of experiments of various types. The GaoKao Natural Science section is composed of physics, chemistry, biology but not earth science, which is different from the CSAT. GaoKao requires basic understanding or the observation ability to reasoning, the complex thinking ability, especially emphasized on the experiment ability. The range of possible questions is in the examination outline, not the curriculum, and the ratio of questions from the University level is high. In the analysis of the behavioral domain, the ratios of the understanding and application items is higher than the CSAT, and inquiry items is lower, but the inquiry items are deeper. In case of the ratio of the expected correct answer, National Test 1 and National Test 2 is similar, but the difficult items or about 20~39% of the test is 4~5 times to that of the CSAT, making the GaoKao very difficult. The peculiar characteristics of GaoKao is the emphasis on the experiment, and even though the practical items is of lower ratio, they are very useful in life.

• English Language & Literature Teaching
• /
• v.18 no.1
• /
• pp.171-195
• /
• 2012

While noticing that there is insufficient interest in the assessment of EFL learners' vocabulary levels or sizes, the researchers developed two tests identical in form (Forms A and B) to assess the lexical knowledge of Korean university learners at the $1^{st}{\sim}10^{th}$ 1,000 word bands by adapting a pre-established vocabulary levels test (VLT). Of equal concern was to investigate if the VLT was equally a valid and reliable instrument to be used on measuring the lexical knowledge of EFL learners. The participants were 804 university freshmen enrolled in a General Education English Course from four different colleges. The learners were asked to respond to either Form A or B. While scores generally fell towards the lower frequency bands, multiple regression found the Korean College Scholastic Ability Test (CSAT) to be a significant variable for predicting the learners' vocabulary sizes. From a methodological perspective, however, noticeable differences between Forms A and B could be found with item response theory analysis. The findings of the study provide suggestions on how future VLT for testing EFL learners may have to be redesigned.

• Journal of the Korean earth science society
• /
• v.22 no.6
• /
• pp.469-479
• /
• 2001

This study investigated the trends in the number of applicants and mean score and applicants’ responses on the test items of Earth Science in the College Scholastic Ability Test (CSAT) implemented for 3 years (1999-2001). The percentage of applicants of science track were 43.14% in 1995, but reduced through 29.5% in 2001 to 26.92% in 2002 CSAT For elective subject, about 22% of science track students applied to Earth Science II which ranked third to Biology II and Chemistry II. In 1999, test items were developed to have the expected difficulty 40 ${\sim}$ 59% (6 items) to 60 ${\sim}$ 79% (10 items). But in 2001 every 16 items were developed to have difficulty 60 ${\sim}$ 79%, which was caused by the policy of so called ‘easy CSAT’. Thus the mean score of ‘Earth Science II’ was increased from 50.26 in 1999 through 64.47 in 2000, to 67.58 in 2001. Applicants were generally very good at solving test items focusing on process skills only and familar items but poor at solving test items related to the motion of the earth and planets and sea wave, especially items calling two or more concepts. Thus special measures to cope with the decrease in applicants of science track should be provided. And it is recommended to develop test items with wider range of difficulty and to reduce test items calling process skills only. And special consideration should be given to teaching the content area with poor achievement and high actual difficulty compared to the expected.

• The Korean Journal of Applied Statistics
• /
• v.32 no.3
• /
• pp.423-434
• /
• 2019

An important issues in Korea society in 2018 was the revision of the university entrance examination system. Among the discussions, in order to grasp what the issue of admission officer system is, attention was focused on the function of media such as monitoring and criticism as well as the tried topic analysis of related news articles. As a result, the reorganization of the College Scholastic Ability Test (CSAT) was derived and showed the sensitivity of Korean society towards the CSAT. Topics directly related to the admission officer system were the selection factor and fairness of the university entrance examination system in relation to the selection factor.

• Communications of Mathematical Education
• /
• v.37 no.2
• /
• pp.233-256
• /
• 2023

This study conducted foundational research to derive ways to use ChatGPT in mathematics education by analyzing ChatGPT's responses to questions from the National Assessment of Educational Achievement (NAEA) and the College Scholastic Ability Test (CSAT). ChatGPT, a generative artificial intelligence model, has gained attention in various fields, and there is a growing demand for its use in education as the number of users rapidly increases. To the best of our knowledge, there are very few reported cases of educational studies utilizing ChatGPT. In this study, we analyzed ChatGPT 3.5 responses to questions from the three-year National Assessment of Educational Achievement and the College Scholastic Ability Test, categorizing them based on the percentage of correct answers, the accuracy of the solution process, and types of errors. The correct answer rates for ChatGPT in the National Assessment of Educational Achievement and the College Scholastic Ability Test questions were 37.1% and 15.97%, respectively. The accuracy of ChatGPT's solution process was calculated as 3.44 for the National Assessment of Educational Achievement and 2.49 for the College Scholastic Ability Test. Errors in solving math problems with ChatGPT were classified into procedural and functional errors. Procedural errors referred to mistakes in connecting expressions to the next step or in calculations, while functional errors were related to how ChatGPT recognized, judged, and outputted text. This analysis suggests that relying solely on the percentage of correct answers should not be the criterion for assessing ChatGPT's mathematical performance, but rather a combination of the accuracy of the solution process and types of errors should be considered.

• Korea Real Estate Review
• /
• v.27 no.4
• /
• pp.63-76
• /
• 2017

The purpose of this study is to analyze the effect of the results of a nationwide academic evaluation of middle schools and high schools on apartment prices in Ulsan City by using a hedonic pricing model. The results of the middle school and high school achievement test, the College Scholastic Ability Test (CSAT) score for high school, the national united evaluation score, and the number of successful applicants to prestigious universities have a significant effect on the apartment price formation with a positive relationship. In addition, different kinds of academic evaluation score have asymmetric effects on apartment price determination. The results of the high school achievement evaluation are more important than the results of the middle school achievement evaluation in the apartment price determination. Among the achievement evaluation results, the ratio of the students with the higher education level is more important than the ratio of the students with the lower basic education level. Furthermore, the CSAT score for Natural Sciences is more important than the CSAT score for the Humanities course.

• School Mathematics
• /
• v.19 no.2
• /
• pp.369-384
• /
• 2017

Current Mathematics of CSAT(College Scholastic Ability Test) faced with preparing the test system and test items according to the new curriculum. The discuss about how to construct the items and what form of items should be set was not conducted enough. To accord with these requirements, in this study, mathematics exams for university entrance in Taiwan are investigated. We look into General Scholastic Ability Test(GSAT) and Advanced Subjects Test(AST) in Taiwan. Those exams are analyzed in terms of exam system, mathematical contents, types of items, and so on. And then on the basis of this, we discuss implications on mathematics assessment type and contents, further mathematics learning.