• Title/Summary/Keyword: validation rating

Search Result 98, Processing Time 0.03 seconds

Evaluation of Regression Models in LOADEST to Estimate Suspended Solid Load in Hangang Waterbody (한강수계에서의 부유사 예측을 위한 LOADEST 모형의 회귀식의 평가)

  • Park, Youn Shik;Lee, Ji Min;Jung, Younghun;Shin, Min Hwan;Park, Ji Hyung;Hwang, Hasun;Ryu, Jichul;Park, Jangho;Kim, Ki-Sung
    • Journal of The Korean Society of Agricultural Engineers
    • /
    • v.57 no.2
    • /
    • pp.37-45
    • /
    • 2015
  • Typically, water quality sampling takes place intermittently since sample collection and following analysis requires substantial cost and efforts. Therefore regression models (or rating curves) are often used to interpolate water quality data. LOADEST has nine regression models to estimate water quality data, and one regression model needs to be selected automatically or manually. The nine regression models in LOADEST and auto-selection by LOADEST were evaluated in the study. Suspended solids data were collected from forty-nine stations from the Water Information System of the Ministry of Environment. Suspended solid data from each station was divided into two groups for calibration and validation. Nash-Stucliffe efficiency (NSE) and coefficient of determination ($R_2$) were used to evaluate estimated suspended solid loads. The regression models numbered 1 and 3 in LOADEST provided higher NSE and $R_2$, compared to the other regression models. The regression modes numbered 2, 5, 6, 8, and 9 in LOADEST provided low NSE. In addition, the regression model selected by LOADEST did not necessarily provide better suspended solid estimations than the other regression models did.

The Development and Validation of the Korean Strength Scale (한국인 강점 척도의 개발 및 타당화)

  • Jung, Young-Eun;Lee, Ji-Eun;Han, You;Choi, Jeong-Woo;Baek, Kyoung Hee;Park, Joo-Eon;Min, Jung-Ah;Chae, Jeong-Ho
    • Anxiety and mood
    • /
    • v.9 no.1
    • /
    • pp.45-53
    • /
    • 2013
  • Objectives : The purpose of this study was to develop the Korean Strength Scale and to examine its validity and reliability. Methods : The Korean Strength Scale is a self-report questionnaire that measures 25 valued strengths and is comprised of 124 items ; each item had a 0-5 rating on a 6-point scale. In order to test validity and reliability, data were collected from 355 adults. The measures included the Korean Strength Scale, HEXACO Personality Inventory (HEXACO-PI), Satisfaction with life scale (SWLS), Positive Affect and Negative Affect Schedule (PANAS), and Orientations to Happiness Questionnaire (OHQ). Results : The resulting exploratory factor analysis of the Korean Strength Scale suggested 4 factor structures. The Korean Strength Scale was shown to have acceptable psychometric properties, including acceptable internal-consistency reliabilities, factorial validity, and high convergent correlations. Conclusion : Although there is room on improvement for some facet scales, the Korean Strength Scale appears to be a useful tool for assessing an individual's signature strengths.

Reliability and Validity of the Korean Translation of Quantitative Checklist for Autism in Toddlers: A Preliminary Study

  • Park, Subin;Won, Eun-Kyung;Lee, Ji Hyun;Yoon, Soyoung;Park, Eun Jin;Kim, Yeni
    • Journal of the Korean Academy of Child and Adolescent Psychiatry
    • /
    • v.29 no.2
    • /
    • pp.80-85
    • /
    • 2018
  • Objectives: We aimed to assess the test-retest reliability, internal consistency, and validity of the Korean version of the Quantitative Checklist for Autism in Toddlers (Q-CHAT). Methods: The Korean version of the Q-CHAT and the Korean version of the Child Behavior Checklist (CBCL) 1.5-5 were completed by parents of 24 toddlers and preschoolers with autism spectrum disorder (ASD) and 80 unselected toddlers and preschoolers. Parents of the ASD group also completed the Social Communication Questionnaire (SCQ), and Childhood Autism Rating Scale (CARS) scores were obtained from medical records. Results: The ASD group scored higher on the Q-CHAT than the unselected group. The Cronbach's alpha coefficient of the Q-CHAT was 0.658, and test-retest reliability was calculated to be 0.836. The estimated area under the curve was 0.793. The total scores of the Q-CHAT in the ASD group demonstrated significant positive correlations with findings regarding pervasive development problems in the CBCL, SCQ, and CARS. A total score of 33.5 may be a useful cutoff point to use when identifying toddlers at risk of ASD. Conclusion: The Korean version of the Q-CHAT has good reliability and validity and can be used as a screening tool in order to identify toddlers and preschool children at risk of ASD.

Quantitative Assessment of Tremor in PD Using a Wearable System on Both Hands (양손에서 웨어러블 시스템을 이용한 파킨슨병의 정량적 진전 평가)

  • Lee, Hongji;Kim, Sangkyong;Kim, Hanbyul;Jeon, Hyoseon;Park, Hyeyoung;Jung, Yujin;Kim, Jeonghwan;Jeon, Beomseok;Park, Kwangsuk
    • Journal of Biomedical Engineering Research
    • /
    • v.35 no.4
    • /
    • pp.81-86
    • /
    • 2014
  • One of the methods for Parkinson's disease(PD) tremor evaluation is the Clinical Tremor Rating Scale(CTRS). However, the method has some limitations that clinician ratings can vary because the scores are subjectively rated. In addition, most researches usually collected data measured on the more affected arm. In this study, we developed a portable wearable system(SNUMAP system) for measuring PD tremor. The SNUMAP system captures 3-dimensional motion using tri-accelerometer and tri-gyroscope on finger and wrist. 40 PD patients participated in resting tremor and postural tremor tasks, while wearing the system on both hands simultaneously. Estimated tremor scores from Leave-One-Out Cross Validation for regression were highly correlated to the average clinician CTRS scores for rest tremor($r^2$ = 0.87, RMSE = 0.48) and postural tremor($r^2$ = 0.82, RMSE = 0.48). Therefore, the quantitative assessment model can improve treatment of PD patients.

Development and Validation of Questionnaire for Chronic Fatigue Syndrome (CFS) Diagnosis Based on Systemic Exertion Intolerance Disease (SEID) Criteria (전신성 활동불능증(Systemic Exertion Intolerance Disease) 진단 기준을 바탕으로 한 만성 피로 증후군(Chronic Fatigue Syndrome) 진단 설문지 개발 및 신뢰도 평가)

  • Lim, Eun-jin;Son, Chang-gue;Jang, Eun-su
    • The Journal of Internal Korean Medicine
    • /
    • v.41 no.3
    • /
    • pp.293-305
    • /
    • 2020
  • Purpose: This study aimed to develop a questionnaire for the diagnosis of chronic fatigue syndrome (CFS) designed based on the systematic exertion intolerance disorder (SEID) criteria, and to validate the reliability of the questionnaire. Methods: A literature search on questionnaires for CFS diagnosis was conducted to develop a SEID questionnaire (SEID-Q27), followed by a pilot survey to identify the reliability of the questionnaire. Adults (Daejeon university personnel) with a Chalder fatigue scale (CFQ) score ≥15 were invited for the survey. We commenced the survey in November 2019 with a two weeks of interval for the test and retest method. The reliability of the questionnaire was investigated in three angles: 1. Cronbach's α, 2. correlations (r) of the questions, numerical rating scale (NRS), and visual analog scale (VAS), and 3. kappa (k) analysis. Results: Among the total 275 adults registered, 55 (20%) participants with a CFQ score ≥15 were invited, and 31 (11%) [15 male, 16 female] completed the questionnaire. The total Cronbach's α was 0.944 for the test and 0.949 for the retest. The reliability (r) of questions by CFQ score (≥15, ≥18, ≥20) ranged from 0.533-0.928 (p <0.05), and the r score of the NRS and VAS were the highest in CFQ scores ≥20, at 0.933 (p<0.001). The agreement rate of the SEID-Q27 between the test and retest was 87% (kappa k=0.743). Conclusions: The SEID-Q27 seems to be reliable. Further studies are needed to measure the validity of the tool and the cutoff point.

A Study for validation of the parental satisfaction scale for child care centers (어린이집 이용만족도 척도 타당화 연구)

  • Shin, Nary;Ahn, Jae-jin
    • Journal of the Korean Society of Child Welfare
    • /
    • no.36
    • /
    • pp.231-259
    • /
    • 2011
  • The purpose of this study is to examine the validity and reliability of the Parental Satisfaction Scale for Child Care Centers, which modified the Client Satisfaction Scale (CSS; Kim, 2009). The subjects of the study were 652 parents of children who were between the ages of one to five and who attended child care centers. The five-point rating scale consisted of 21 items with two sub-dimensions. The results from an exploratory factor analysis identified that there are two dimensions in this scale. Thus, in terms of the face validity of this scale, a two-dimensional scale with 21 items is found to be appropriate. However, the items that consist of the two sub-dimensions are found to be different from that of CSS. The concurrent validity and internal consistency reliability of the revised scale are relatively high. Also, the mean between the upper and lower groups with regard to item discrimination show significant difference. We conclude that the original CSS with minor revisions can be used as a valid and reliable instrument to assess parental satisfaction on child care centers.

Effect of tunnel fire: Analysis and remedial measures

  • Choubey, Bishwajeet;Dutta, Sekhar C.;Kumar, Virendra
    • Structural Engineering and Mechanics
    • /
    • v.80 no.6
    • /
    • pp.701-709
    • /
    • 2021
  • The paper aims at improving the understanding and mitigating the effects of tunnel fires that may breakout due to the burning fuel and/or explosion within the tunnel. This study particularly focuses on the behavior of the commonly used horse shoe geometry of tunnel systems. The problem has been obtained using an adequate well-established program incorporating the Lagrangian approach. A transient-thermo-coupled static structural analysis is carried out. The effects of radiation and convection to the outer walls of the tunnel is studied. The paper also presents the impact of the hazard on the structural integrity of the tunnel. A methodology is proposed to study the tunnel fire using a model which uses equivalent steel sheet to represent the presence of reinforcements to improve the computational efficiency with adequate validation. A parametric study has been carried out and the effect of suitable lining property for mitigating the fire hazard is arrived at. Detailed analysis is done for the threshold limits of the properties of the lining material to check if it is acceptable in all aspects for the integrity of the tunnel. The study may prove useful for developing insights for ensuring tunnel fire safety. To conduct such studies experimentally are tremendously costly but are required to gain confidence. But, scaled models, as well as loading and testing conditions, cannot be studied by many trials experimentally as the cost will shoot up sharply. In this context, the results obtained from such computational studies with a feasible variation of various combinations of parameters may act as a set of guidelines to freeze the adequate combination of various parameters to conduct one or two costly experiments for confidence building.

Comparing the effects of letter-based and syllable-based speaking rates on the pronunciation assessment of Korean speakers of English (철자 기반과 음절 기반 속도가 한국인 영어 학습자의 발음 평가에 미치는 영향 비교)

  • Hyunsong Chung
    • Phonetics and Speech Sciences
    • /
    • v.15 no.4
    • /
    • pp.1-10
    • /
    • 2023
  • This study investigated the relative effectiveness of letter-based versus syllable-based measures of speech rate and articulation rate in predicting the articulation score, prosody fluency, and rating sum using "English speech data of Koreans for education" from AI Hub. We extracted and analyzed 900 utterances from the training data, including three balanced age groups (13, 19, and 26 years old). The study built three models that best predicted the pronunciation assessment scores using linear mixed-effects regression and compared the predicted scores with the actual scores from the validation data (n=180). The correlation coefficients between them were also calculated. The findings revealed that syllable-based measures of speech and articulation rates were more effective than letter-based measures in all three pronunciation assessment categories. The correlation coefficients between the predicted and actual scores ranged from .65 to .68, indicating the models' good predictive power. However, it remains inconclusive whether speech rate or articulation rate is more effective.

A Multimodal Profile Ensemble Approach to Development of Recommender Systems Using Big Data (빅데이터 기반 추천시스템 구현을 위한 다중 프로파일 앙상블 기법)

  • Kim, Minjeong;Cho, Yoonho
    • Journal of Intelligence and Information Systems
    • /
    • v.21 no.4
    • /
    • pp.93-110
    • /
    • 2015
  • The recommender system is a system which recommends products to the customers who are likely to be interested in. Based on automated information filtering technology, various recommender systems have been developed. Collaborative filtering (CF), one of the most successful recommendation algorithms, has been applied in a number of different domains such as recommending Web pages, books, movies, music and products. But, it has been known that CF has a critical shortcoming. CF finds neighbors whose preferences are like those of the target customer and recommends products those customers have most liked. Thus, CF works properly only when there's a sufficient number of ratings on common product from customers. When there's a shortage of customer ratings, CF makes the formation of a neighborhood inaccurate, thereby resulting in poor recommendations. To improve the performance of CF based recommender systems, most of the related studies have been focused on the development of novel algorithms under the assumption of using a single profile, which is created from user's rating information for items, purchase transactions, or Web access logs. With the advent of big data, companies got to collect more data and to use a variety of information with big size. So, many companies recognize it very importantly to utilize big data because it makes companies to improve their competitiveness and to create new value. In particular, on the rise is the issue of utilizing personal big data in the recommender system. It is why personal big data facilitate more accurate identification of the preferences or behaviors of users. The proposed recommendation methodology is as follows: First, multimodal user profiles are created from personal big data in order to grasp the preferences and behavior of users from various viewpoints. We derive five user profiles based on the personal information such as rating, site preference, demographic, Internet usage, and topic in text. Next, the similarity between users is calculated based on the profiles and then neighbors of users are found from the results. One of three ensemble approaches is applied to calculate the similarity. Each ensemble approach uses the similarity of combined profile, the average similarity of each profile, and the weighted average similarity of each profile, respectively. Finally, the products that people among the neighborhood prefer most to are recommended to the target users. For the experiments, we used the demographic data and a very large volume of Web log transaction for 5,000 panel users of a company that is specialized to analyzing ranks of Web sites. R and SAS E-miner was used to implement the proposed recommender system and to conduct the topic analysis using the keyword search, respectively. To evaluate the recommendation performance, we used 60% of data for training and 40% of data for test. The 5-fold cross validation was also conducted to enhance the reliability of our experiments. A widely used combination metric called F1 metric that gives equal weight to both recall and precision was employed for our evaluation. As the results of evaluation, the proposed methodology achieved the significant improvement over the single profile based CF algorithm. In particular, the ensemble approach using weighted average similarity shows the highest performance. That is, the rate of improvement in F1 is 16.9 percent for the ensemble approach using weighted average similarity and 8.1 percent for the ensemble approach using average similarity of each profile. From these results, we conclude that the multimodal profile ensemble approach is a viable solution to the problems encountered when there's a shortage of customer ratings. This study has significance in suggesting what kind of information could we use to create profile in the environment of big data and how could we combine and utilize them effectively. However, our methodology should be further studied to consider for its real-world application. We need to compare the differences in recommendation accuracy by applying the proposed method to different recommendation algorithms and then to identify which combination of them would show the best performance.

Validation of the coach-athlete relationship scale of amateur golf players: Rasch rating scale model (아마추어 골프 선수를 위한 코치-선수 관계 척도의 타당화: Rasch 평정척도 모형 적용)

  • Kim, Sae Hyung;Choi, Jae Il;Lee, Jun Woo
    • Journal of the Korean Data and Information Science Society
    • /
    • v.24 no.6
    • /
    • pp.1319-1329
    • /
    • 2013
  • The purpose of this research was to develop and validate the coach-athlete relationship scale suitable to amateur golf players by applying the Rasch rating scale model. As the coach-athlete relationship scale, the Korean form of scale developed by Kim and Park (2008), which was revised based on the evidence on the basis of inspection contents, was used to conduct a survey on 217 amateur golf athletes. And the unidimensionality, which is the basic assumption of the Rasch model, was verified using the WINSTEPS program, and the appropriateness of the item category was established through the step calibration. The goodness of fit of each question was tested through the goodness-of-fit index and the differential item functioning (DIF) was estimated according to the golf career. When the goodness-of-fit index estimated for each question was 1.30 or more it was judged unfit and the significance level in the analysis was all set as.05. The results of the analysis showed that the measures variance explained by the Rasch measurement model was more (33.7%) than 20%, so the unidimensionality assumptions of the 11 questions (..hospitable posture when my coach is teaching) were satisfied. The result of analyzing the item category (7 scale) with step calibration was found to be unfit, but in the result of reanalyzing by rescoring into a 5-point scale, it was found to be fit. Particularly, in the result of estimating the goodness-of-fit using the systematized item category (5 scale), Question 10 (...my best when my coach is teaching) and Question 11 were found to be unfit, and as a result of estimating the differential functioning item according to golf career, Question 11 was found to be unevenly differentiated according to golf career. So the 5-point scale of Question 9 after eliminating the two questions which were unfit and differentiated was validated to be the coach-athlete relationship scale suitable to amateur golf athletes.