• Title/Summary/Keyword: generalizability theory

Search Result 26, Processing Time 0.022 seconds

Assessing the Validity of the Preclinical Objective Structured Clinical Examination Using Messick's Validity Framework (Messick의 타당도 틀을 활용한 임상실습 전 실기시험의 타당도 평가)

  • Lee, Hye-Yoon;Yune, So-Jung;Lee, Sang-Yeoup;Im, Sunju
    • Korean Medical Education Review
    • /
    • v.23 no.3
    • /
    • pp.185-193
    • /
    • 2021

  • Students must be familiar with clinical skills before starting clinical practice to ensure patients' safety and enable efficient learning. However, performance is mainly tested in the third or fourth years of medical school, and studies using the validity framework have not been reported in Korea. We analyzed the validity of a performance test conducted among second-year students classified into content, response process, internal structure, relationships with other variables, and consequences according to Messick's framework. As results of the analysis, content validity was secured by developing cases according to a pre-determined blueprint. The quality of the response process was controlled by training and calibrating raters. The internal structure showed that (1) reliability by generalizability theory was acceptable (coefficients of 0.724 and 0.786, respectively, for day 1 and day 2), and (2) the relevant domains had proper correlations, while the clinical performance examination (CPX) and objective structured clinical examination (OSCE) showed weaker relationships. OSCE/CPX scores were correlated with other variables, especially grade point average and oral structured exam scores. The consequences of this assessment were (1) making students learn clinical skills and study themselves, while causing too much stress for students due to lack of motivation; (2) reminding educators of the need to apply practical teaching methods and to give feedback on the test results; and (3) providing an opportunity for faculty to consider developing support programs. It is necessary to develop the blueprint more precisely according to students' level and to verify the validity of the response process with statistical methods.

  • https://doi.org/10.17496/kmer.2021.23.3.185 인용 PDF KSCI

A Comparison of Item Characteristics and Test Information Between the K-MMSE~2:SV and K-MMSE

  • Jihyang Kim;Seungmin Jahng;SangYun Kim;Yeonwook Kang
    • Dementia and Neurocognitive Disorders
    • /
    • v.23 no.3
    • /
    • pp.117-126
    • /
    • 2024

  • Background and Purpose: The Korean-Mini Mental State Examination, 2nd edition (K-MMSE~2) was recently released. This study aimed to determine whether the K-MMSE~2: Standard Version (K-MMSE~2:SV) had the same test characteristics as the K-MMSE. Methods: A total of 1,514 healthy community-based participants aged 19 to 90 years were administered the K-MMSE~2:SV Blue Form along with the language items from the K-MMSE. The item and test characteristics and test information for the K-MMSE~2:SV and K-MMSE were compared using Item Response Theory analysis. Results: Item discriminations for the K-MMSE~2:SV and K-MMSE were above the moderate range for all items except Recall. Most of the items on the K-MMSE~2:SV and K-MMSE had item category difficulty in the very easy or easy range. The test information curve (TIC) showed that the K-MMSE~2:SV and K-MMSE provide almost the same amount of information (27.86 vs. 28.44), with both tests providing the most information at an ability level of -1.57. The generalizability (G) coefficient for the K-MMSE~2:SV and K-MMSE was 0.99. Conclusions: These results indicate that the K-MMSE~2:SV and K-MMSE are equally optimal tests for screening for mild cognitive impairment and early dementia. Given that the amount of test information provided by the two tests was almost identical, the shapes of the TICs were very similar, and the G coefficient was close to 1, we can conclude that the K-MMSE and K-MMSE~2:SV are equivalent tests.

  • https://doi.org/10.12779/dnd.2024.23.3.117 인용 PDF

The Development and Validation of Instrument for Measuring High School Students' STEM Career Motivation (고등학생들을 위한 이공계 진로동기 검사도구 개발 및 타당화)

  • Shin, Sein;Ha, Minsu;Lee, Jun-Ki
    • Journal of The Korean Association For Science Education
    • /
    • v.36 no.1
    • /
    • pp.75-86
    • /
    • 2016

  • The purpose of the present study is to develop and validate an instrument to assess STEM career motivation. We developed 32 items for 7 constructs (i.e. education experience, career value, academic self-efficacy, career self-efficacy, career interest, parents' support, and career motivation) on STEM career motivation based on Social Cognitive Career Theory (SCCT; Lent et al.,1994). 767 first year high school students participated in this study. The items were validated by Messick's framework (1995). In this study, we examined the validity of items in four aspects (i.e. content, substantive, structural and generalizability of validity). Methodologically, we used Rasch analysis, Exploratory factor analysis, confirmative factor analysis based on structural equation modelling. We confirmed that our instrument with 32 items as valid and reliable for measuring the STEM career motivation. In addition, we tested the STEM career motivation model based on SCCT. Our model explained the data well, suggesting that external factors (education experience and parents' support) and cognitive factors (perception of value, self-efficacy and interest) were significantly related to STEM career motivation.

  • https://doi.org/10.14697/jkase.2016.36.1.0075 인용 PDF KSCI

Applying Strategy Group Concept to Program Providers(PP) Industry (PP 산업에 대한 전략집단 개념의 적용)

  • Yeo, Hyun-Chul;Kim, Young-Soo
    • The Journal of the Korea Contents Association
    • /
    • v.11 no.1
    • /
    • pp.357-370
    • /
    • 2011

  • Using strategy group theory, this thesis reviewed the status of program providers analysis and the performances it has made so far, and sought measures to improve its limitations. The constraint of program providers analysis based on existing concept of strategy group is that the strategy group was derived from the statistics, and therefore only applied the characteristics of program provider's channels to the analysis, on account of which a systematic and sophisticated classification as well as generalization of strategy or strategy group were hard to obtain. Moreover, the PP strategy variables used to be selected at the firm level and business level, and in relation with resource and competition scope. In future, more appropriate procedure should be followed to obtain objectivity in selecting variables to avoid controversy over intentionality. The measures in this thesis to improve the study of PP strategy group can be summarized as follows: firstly analysis of variables for strategy group classification should be made to single out key variables which are to be classification criteria. Secondly, variables are to be cross-checked by industry experts to increase generalizability. Thirdly, proxy variables should be sublated, and strategy group model which enables the reflection of subsistent properties of PP industry, and the cognitive perception of the executives(CEO) needs to be established. Fourthly, the concepts of mobility barrier and isolating mechanism should be applied to the classification criteria of strategy group to reveal the gap of performance between different strategy groups. Lastly, chronicle study on PP strategy group should be done to perceive the dynamic changes of PP strategy group.

  • https://doi.org/10.5392/JKCA.2011.11.1.357 인용 PDF KSCI

Development and Implementation of a Process-Based Evaluation Program on School Space Design: Focusing on the Housing Life Area of Home Economics Curriculum in Middle School (학교 공간 디자인을 주제로 한 과정중심평가 프로그램 개발 및 실행: 중학교 가정교과 주생활 영역을 중심으로)

  • Kang, Eun Young;Park, Mi Jeong
    • Journal of Korean Home Economics Education Association
    • /
    • v.32 no.4
    • /
    • pp.81-101
    • /
    • 2020

  • The purpose of this study is to develop and implement a process-based evaluation program with the theme of school space design in the housing area of the middle school home economics. In order to achieve For thispurpose, a process-based evaluation program based on the theme of school space design was developed following the ADDIE instructional design model, and the program was executed to a total of 93 students. A questionnaire survey and in-depth interview were conducted for the evaluation of the program. The results of this study are as follows. First, based on the results of a 2015-Revised Curriculum analysis, a school space design program evaluation plan was established, and two evaluation tasks were developed. Accordingly, scoring criteria were prepared and 8 evaluation materials for students and 2 evaluation materials for teachers were developed. A total of 9 sessions were developed for teaching and learning activities and evaluation-linked operation procedures to perform evaluation tasks. As a result of an expert validity test for the program, all items were verified to be appropriate in content validity and content composition with an average of 3.6 to 4 points (4 points). Second, after conducting the school space design program, a survey on students were conducted, and as a result, all three areas of school space design class, process-based evaluation, interest scored high in average scores of 4.12 to 4.27 out of 5. According to the survey and interview results, the program provided new learning opportunities for school space design, the students were able to reach the suggested achievement goals, and the self-assessment, peer evaluation and teacher feedback positively affected the students during the learning process so that they could reflect on their learning and actively participate in the subsequent learning activity. This study has a limitation in generalizability in that the program was conducted on a limited number of students, and future studies are expected to expand the scope in terms of research participants, evaluation criteria, and school space design classes. This study laid the foundation for theory and practice by developing and implementing a process-based evaluation program for home economics education, and it has contribution in that it suggested the possibility that teachers and students can take the initiatives in school space design, focusing on the housing content elements of home economics.

  • https://doi.org/10.19031/jkheea.2020.12.32.4.81 인용 PDF

The Impact of Market Environments on Optimal Channel Strategy Involving an Internet Channel: A Game Theoretic Approach (시장 환경이 인터넷 경로를 포함한 다중 경로 관리에 미치는 영향에 관한 연구: 게임 이론적 접근방법)

  • Yoo, Weon-Sang
    • Journal of Distribution Research
    • /
    • v.16 no.2
    • /
    • pp.119-138
    • /
    • 2011

  • Internet commerce has been growing at a rapid pace for the last decade. Many firms try to reach wider consumer markets by adding the Internet channel to the existing traditional channels. Despite the various benefits of the Internet channel, a significant number of firms failed in managing the new type of channel. Previous studies could not cleary explain these conflicting results associated with the Internet channel. One of the major reasons is most of the previous studies conducted analyses under a specific market condition and claimed that as the impact of Internet channel introduction. Therefore, their results are strongly influenced by the specific market settings. However, firms face various market conditions in the real worlddensity and disutility of using the Internet. The purpose of this study is to investigate the impact of various market environments on a firm's optimal channel strategy by employing a flexible game theory model. We capture various market conditions with consumer density and disutility of using the Internet.

    shows the channel structures analyzed in this study. Before the Internet channel is introduced, a monopoly manufacturer sells its products through an independent physical store. From this structure, the manufacturer could introduce its own Internet channel (MI). The independent physical store could also introduce its own Internet channel and coordinate it with the existing physical store (RI). An independent Internet retailer such as Amazon could enter this market (II). In this case, two types of independent retailers compete with each other. In this model, consumers are uniformly distributed on the two dimensional space. Consumer heterogeneity is captured by a consumer's geographical location (ci) and his disutility of using the Internet channel (${\delta}_{N_i}$).
    shows various market conditions captured by the two consumer heterogeneities.
    (a) illustrates a market with symmetric consumer distributions. The model captures explicitly the asymmetric distributions of consumer disutility in a market as well. In a market like that is represented in
    (c), the average consumer disutility of using an Internet store is relatively smaller than that of using a physical store. For example, this case represents the market in which 1) the product is suitable for Internet transactions (e.g., books) or 2) the level of E-Commerce readiness is high such as in Denmark or Finland. On the other hand, the average consumer disutility when using an Internet store is relatively greater than that of using a physical store in a market like (b). Countries like Ukraine and Bulgaria, or the market for "experience goods" such as shoes, could be examples of this market condition. summarizes the various scenarios of consumer distributions analyzed in this study. The range for disutility of using the Internet (${\delta}_{N_i}$) is held constant, while the range of consumer distribution (${\chi}_i$) varies from -25 to 25, from -50 to 50, from -100 to 100, from -150 to 150, and from -200 to 200.
    summarizes the analysis results. As the average travel cost in a market decreases while the average disutility of Internet use remains the same, average retail price, total quantity sold, physical store profit, monopoly manufacturer profit, and thus, total channel profit increase. On the other hand, the quantity sold through the Internet and the profit of the Internet store decrease with a decreasing average travel cost relative to the average disutility of Internet use. We find that a channel that has an advantage over the other kind of channel serves a larger portion of the market. In a market with a high average travel cost, in which the Internet store has a relative advantage over the physical store, for example, the Internet store becomes a mass-retailer serving a larger portion of the market. This result implies that the Internet becomes a more significant distribution channel in those markets characterized by greater geographical dispersion of buyers, or as consumers become more proficient in Internet usage. The results indicate that the degree of price discrimination also varies depending on the distribution of consumer disutility in a market. The manufacturer in a market in which the average travel cost is higher than the average disutility of using the Internet has a stronger incentive for price discrimination than the manufacturer in a market where the average travel cost is relatively lower. We also find that the manufacturer has a stronger incentive to maintain a high price level when the average travel cost in a market is relatively low. Additionally, the retail competition effect due to Internet channel introduction strengthens as average travel cost in a market decreases. This result indicates that a manufacturer's channel power relative to that of the independent physical retailer becomes stronger with a decreasing average travel cost. This implication is counter-intuitive, because it is widely believed that the negative impact of Internet channel introduction on a competing physical retailer is more significant in a market like Russia, where consumers are more geographically dispersed, than in a market like Hong Kong, that has a condensed geographic distribution of consumers.
    illustrates how this happens. When mangers consider the overall impact of the Internet channel, however, they should consider not only channel power, but also sales volume. When both are considered, the introduction of the Internet channel is revealed as more harmful to a physical retailer in Russia than one in Hong Kong, because the sales volume decrease for a physical store due to Internet channel competition is much greater in Russia than in Hong Kong. The results show that manufacturer is always better off with any type of Internet store introduction. The independent physical store benefits from opening its own Internet store when the average travel cost is higher relative to the disutility of using the Internet. Under an opposite market condition, however, the independent physical retailer could be worse off when it opens its own Internet outlet and coordinates both outlets (RI). This is because the low average travel cost significantly reduces the channel power of the independent physical retailer, further aggravating the already weak channel power caused by myopic inter-channel price coordination. The results implies that channel members and policy makers should explicitly consider the factors determining the relative distributions of both kinds of consumer disutility, when they make a channel decision involving an Internet channel. These factors include the suitability of a product for Internet shopping, the level of E-Commerce readiness of a market, and the degree of geographic dispersion of consumers in a market. Despite the academic contributions and managerial implications, this study is limited in the following ways. First, a series of numerical analyses were conducted to derive equilibrium solutions due to the complex forms of demand functions. In the process, we set up V=100, ${\lambda}$=1, and ${\beta}$=0.01. Future research may change this parameter value set to check the generalizability of this study. Second, the five different scenarios for market conditions were analyzed. Future research could try different sets of parameter ranges. Finally, the model setting allows only one monopoly manufacturer in the market. Accommodating competing multiple manufacturers (brands) would generate more realistic results.

  • PDF
    • (34141) Korea Institute of Science and Technology Information, 245, Daehak-ro, Yuseong-gu, Daejeon
    Copyright (C) KISTI. All Rights Reserved.