• Analyses & Alternatives
• /
• v.5 no.1
• /
• pp.3-24
• /
• 2021

This paper introduces a multilevel item-response-theory (IRT) model as a unifying model for hypothesis testing using legislative voting data. This paper shows that a probit or logit model is a special type of multilevel IRT model. In particular, it is demonstrated that, when a probit or logit model is applied to multiple votes, it makes unrealistic assumptions and produces incorrect coefficient estimates. The advantages of a multilevel IRT model over a probit or logit model are illustrated with a Monte Carlo experiment and an example from the U.S. House. Finally, this paper provides a practical guide to fitting this model to legislative voting data.

• Journal of Korean Biological Nursing Science
• /
• v.20 no.1
• /
• pp.11-19
• /
• 2018

Purpose: The purposes of this study were to perform items analysis using the classical test theory (CTT) and the item response theory (IRT), and to establish the validity and reliability of the Korean version of pressure ulcer prevention knowledge. Methods: The 26-item pressure ulcer prevention knowledge instrument was translated into Korean, and the item analysis of the 22 items having an adequate content validity index (CVI), was conducted. A total of 240 registered nurses in 2 university hospitals completed the questionnaire. Each item was analyzed applying CTT and IRT according to 2-parameter logistic model. Response alternatives quality, item difficulty and item discrimination were evaluated. For testing validity and reliability, Pearson correlation coefficient and Kuder Richardson-20 (KR-20) were used. Results: Scale CVI was .90 (Item-CVI range= .75-1.00). The total correct answer rate for this study population was relatively low as 52.5%. The quality of response alternatives was found to be relatively good (range= .02-.83). The item difficulty of the questions ranged form .10 to .86 according to CTT and -12.19 to 29.92 according to the IRT. This instrument had 12-low, 2-medium and 8-high item difficulty applying IRT. The values for the item discrimination ranged .04-.57 applying CTT and .00-1.47 applying IRT. And overall internal consistency (KR-20) was .62 and stability (test-retest) was .82. Conclusion: The instrument had relatively weak construct validity, item discrimination according to the IRT. Therefore, the cautious usage of a Korean version of this instrument would be recommended for discrimination because there are so many attractive response alternatives and low internal consistency.

• The Korea Journal of Herbology
• /
• v.37 no.2
• /
• pp.13-21
• /
• 2022

Objectives : The evaluation of academic achievement is pivotal for establishing accurate direction and adequate level of medical education. The purpose of this study was to firstly establish innovative item analysis technique of Item Response Theory (IRT) for analyzing multiple-choice test of herbology in the traditional Korean medicine education which has not been available for the difficulty of test theory and statistical calculation. Methods : The answers of 390 students (2012-2018) to the 14 item herbology test in college of Korean medicine were used for the item analysis. As for the multidimensional analysis of item characteristics, difficulty, discrimination, and guessing parameters along with item-total correlation and percentage of correct answer were calculated using Classical Test Theory (CTT) and IRT. Results : The validity parameters of strong and weak items were illustrated in multiple perspectives. There were 4 items with six acceptable index scores, and 5 items with only one acceptable index score. The item discrimination of IRT was found to have no significant correlation with difficulty and discrimination indices of CTT emphasizing attention of professionals of medical education as for the test credibility. Conclusion : The critical suggestions for the development, utilization and revision of test items in the e-learning and evidence-based Teaching era were made based on the results of item analysis using IRT. The current study would firstly provide foundation for upgrading the quality of Korean medicine education using test theory.

• Communications for Statistical Applications and Methods
• /
• v.15 no.3
• /
• pp.465-481
• /
• 2008

Item response theory(IRT) estimates latent ability of a subject based on the property of item and item parameters using item characteristics curve(ICC) of each item case. The initial value and another problems occurs when we try to estimate item parameters of IRT(e.g. the maximum likelihood estimate). Thus, we propose the asymptotic approximation method(AAM) to solve the above mentioned problems. We notice that the proposed method can be thought as an alternative to estimate item parameters when we have small size of data or need to estimate items with local fluctuations. We developed 'Any Assess' and tested reliability of the system result by simulating a practical use possibility.

• Communications for Statistical Applications and Methods
• /
• v.8 no.1
• /
• pp.193-207
• /
• 2001

When designing a test set, we need to consider constraints on items that are deemed important by item developers or test specialists. The constraints are essentially on the components of the test domain or abilities relevant to a given test set. And so if the test domain could be represented in a more refined form, test construction would be made in a more efficient way. We assume that relationships among task abilities are representable by a causal model and that the item response theory (IRT) is not fully available for them. In such a case we can not apply traditional item selection methods that are based on the IRT. In this paper, we use entropy as an uncertainty measure for making inferences on task abilities and developed an optimal item selection algorithm which reduces most the entropy of task abilities when items are selected from an item pool.

• Journal of Internet Computing and Services
• /
• v.8 no.6
• /
• pp.127-133
• /
• 2007

The u-learning environment has been developed through a number of iterations, and has now been formally evaluated, through analysis of student learning results and the use of quantitative and qualitative measures, Generally, for advance learning effect and analysis of student learning results, the most learning system be use to the item analysis method. But, nowadays, it has using the IRT(Item Response Theory) instead of the item analysis method, The IRT adopts explicit models for the probability of each possible response to a test. Therefore, I proposed the lightweight component based u-learning system using the IRT. Applied device of u-learning is PDA which is in Windows mobile 5.0 environments.

• Physical Therapy Korea
• /
• v.19 no.3
• /
• pp.124-131
• /
• 2012

Most conventional instruments measuring disability rely on total score by simply adding individual item responses, which is dependent on the items chosen to represent the underlying construct (test-dependent) and a test statistic, such as coefficient alpha for the estimate of reliability, varying from sample to sample (sample-dependent). By contrast, item response theory (IRT) method focuses on the psychometric properties of the test items instead of the instrument as a whole. By estimating probability that a respondent will select a particular rating for an item, item difficulty and person ability (or disability) can be placed on same linear continuum. These estimates are invariant regardless of the item used (test-free measurement) and the ability of sample applied (sample-free measurement). These advantages of IRT allow the creation of invariantly calibrated large item banks that precisely discriminate the disability levels of individuals. Computer adaptive testing (CAT) method often requiring a testing algorithm promise a means for administering items in a way that is both efficient and precise. This method permits selectively administering items that are closely matched to the ability level of individuals (measurement precision) and measuring the ability without the loss of precision provided by the full item bank (measurement efficiency). These measurement properties can reasonably be achieved using IRT and CAT method. This article aims to investigate comprehensive overview of the existing disability instrument for back pain and to inform physical therapists of an alternative innovative way overcoming the shortcomings of conventional disability instruments. An understanding of IRT and CAT method will equip physical therapist with skills in interpreting the measurement properties of disability instruments developed using the methods.

• Physical Therapy Korea
• /
• v.26 no.3
• /
• pp.106-114
• /
• 2019

Background: It is difficult to assess whether community-dwelling frail older adults may remain pre-frail status or improve into a robust state without being directly checked by health care professionals. The health information perceived by older adults is considered to be one of best sources of potential concerns in older adult population. An online measurement system combined with item response theory (IRT) and computer adaptive testing (CAT) methods is likely to become a realistic approach to remotely monitor physical activity status of frail older adults. Objects: This article suggests an approach to provide a precise and efficient means of measuring physical activity levels of community-dwelling frail older adults. Methods: Article reviews were reviewed and summarized. Results: In comparison to the classical test theory (CTT), the IRT method is empirically aimed to focus on the psychometric properties of individual test items in lieu of the test as a whole. These properties allow creating a large item pool that can capture the broad range of physical activity levels. The CAT method administers test items by an algorithm that select items matched to the physical activity levels of the older adults. Conclusion: An online measurement system combined with these two methods would allow adequate physical activity measurement that may be useful to remotely monitor the activity level of community-dwelling frail older adults.

• International Journal of Contents
• /
• v.5 no.1
• /
• pp.51-55
• /
• 2009

In this paper, we present a structure of personalized e-Learning system to study for a test formalized by uniform multiple-choice using on/off line mixed estimations as is the case of Driver :s License Test in Korea. Using the system a candidate can study toward the license through the Internet (and/or mobile instruments) within the personalized concept based on IRT(item response theory). The system accurately estimates user's ability parameter and dynamically offers optimal evaluation problems and learning contents according to the estimated ability so that the user can take possession of the license in shorter time. In order to establish the personalized e-Learning concepts, we build up 3 databases and 2 agents in this system. Content DB maintains learning contents for studying toward the license as the shape of objects separated by concept-unit. Item-bank DB manages items with their parameters such as difficulties, discriminations, and guessing factors, which are firmly related to the learning contents in Content DB through the concept of object parameters. User profile DB maintains users' status information, item responses, and ability parameters. With these DB formations, Interface agent processes user ID, password, status information, and various queries generated by learners. In addition, it hooks up user's item response with Selection & Feedback agent. On the other hand, Selection & Feedback agent offers problems and content objects according to the corresponding user's ability parameter, and re-estimates the ability parameter to activate dynamic personalized learning situation and so forth.

• Communications for Statistical Applications and Methods
• /
• v.23 no.6
• /
• pp.531-541
• /
• 2016

Cerebral atrophy affects the brain and is a common feature of patients with mild cognitive impairment or Alzheimer's diseases. It is evaluated by the radiologist or reader based on patient's history, age and the space between the brain and the skull as indicated by magnetic resonance (MR) images. A total of 70 patients were scanned in the supine and prone positions before three radiologist assessed their atrophy level. This study examined the radiologist's assessment of the cerebral atrophy level using a graded response model of item response theory (IRT). A graded response model (GRM) is fitted to our data and then item-fit and person-fit statistics are evaluated to assess the fitted model. Our analysis found that the cerebral atrophy level is better discriminated by readers in the prone position because all item slopes were greater than 2 at this position, versus the supine position where all the slope parameters were less than 1. However, the thresholds are very similar for the first reader and are quite different for the second and third readers because the scanning position affects readers differently as the category threshold estimates vary considerably between the readers..