• Title/Summary/Keyword: Meta-Evaluation

Search Result 346, Processing Time 0.031 seconds

The Feature of the Program components in the Meta Analysis Research : Evidence Based Program Development Perspective (메타분석연구에서 나타난 프로그램 구성요소의 실태 : 증거기반 프로그램 개발의 관점에서)

  • Seo, In Hae;Kong, Gye Soon
    • Korean Journal of Social Welfare Studies
    • /
    • v.49 no.3
    • /
    • pp.247-275
    • /
    • 2018
  • In the absence of a research study on meta-analysis in terms of program development, the purpose of the study is to analyze the contents of the meta-analysis research studies which has been conducted for 18 years, and is to identify the level of program component evidence for the development of social work program. In order to achieve these purposes, the study analyzed the feature and usefulness of the 110 meta-analysis studies(5,781 program evaluation studies)published from 2010 to June 2017 in major academic journals related to the areas of the social welfare, psychology, counseling and health. The major findings are as follows. The 110 meta-analysis studies tended to narrow down the scope of the population, problems, and program types, but they also included a lot of heterogeneous types. In the statistical methods, there were relatively few studies to explain the factors behind the heterogeneity of program effectiveness. In addition, researchers tended to select program components arbitrarily with bias on specific components. The important program components with the statistical validity are as follows; the age of the subjects, the severity of the problem, the expertise of the providers, and the strength and activities of the intervention, The academic meanings of the study results was discussed, and the direction of future research was presented to increase the usefulness of the metaanalysis for program development.

Reporting Qualitative Research of Systematic Review in the Journal of Korean Medicine Rehabilitation According to Preferred Reporting Items for Systematic Reviews and Meta-Analyses 2020 Guidelines (PRISMA 2020 지침에 근거한 한방재활의학과학회지 체계적 문헌고찰 보고의 질 평가 연구)

  • Na, Hyeon-Uk;Park, Shin-Hyeok;Woo, Hyeon-Jun;Han, Yun-Hee;Geum, Ji-Hye;Lee, Jung-Han;Ha, Won-Bae
    • Journal of Korean Medicine Rehabilitation
    • /
    • v.32 no.3
    • /
    • pp.85-107
    • /
    • 2022
  • Objectives The purpose of this study was to assess the reporting quality of systematic reviews and meta-analyses in the Journal of Korean Medicine Rehabilitation (JKMR) using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) 2020 guidelines. Methods Systematic reviews and meta-analyses in JKMR, published from January 1991 to January 2022, were selected by searching the Korean studies Information Service System and JKMR homepage. Two independent researchers searched and selected systematic reviews and meta-analyses and evaluated the reporting quality of abstracts and main texts using the PRISMA 2020 guidelines. Results Of 1,515 articles, 39 systematic reviews were finally included for assessment. Evaluation of abstracts resulted in 2 studies rated as high, 11 studies rated as moderate, and 26 studies rated as low. A maximum of 83.3% and a minimum of 25.0% of the items were reported in the abstracts. Evaluation of the manuscripts resulted in no studies rated as high, 14 studies rated as moderate, and 25 studies rated as low. A maximum of 67.9% and a minimum of 34.1% of the items were reported in the manuscripts. Conclusions To improve the quality of systematic reviews published in JKMR, it is necessary to conduct systematic reviews based on the PRISMA 2020 guidelines.

The Effect of Meta-Features of Multiclass Datasets on the Performance of Classification Algorithms (다중 클래스 데이터셋의 메타특징이 판별 알고리즘의 성능에 미치는 영향 연구)

  • Kim, Jeonghun;Kim, Min Yong;Kwon, Ohbyung
    • Journal of Intelligence and Information Systems
    • /
    • v.26 no.1
    • /
    • pp.23-45
    • /
    • 2020
  • Big data is creating in a wide variety of fields such as medical care, manufacturing, logistics, sales site, SNS, and the dataset characteristics are also diverse. In order to secure the competitiveness of companies, it is necessary to improve decision-making capacity using a classification algorithm. However, most of them do not have sufficient knowledge on what kind of classification algorithm is appropriate for a specific problem area. In other words, determining which classification algorithm is appropriate depending on the characteristics of the dataset was has been a task that required expertise and effort. This is because the relationship between the characteristics of datasets (called meta-features) and the performance of classification algorithms has not been fully understood. Moreover, there has been little research on meta-features reflecting the characteristics of multi-class. Therefore, the purpose of this study is to empirically analyze whether meta-features of multi-class datasets have a significant effect on the performance of classification algorithms. In this study, meta-features of multi-class datasets were identified into two factors, (the data structure and the data complexity,) and seven representative meta-features were selected. Among those, we included the Herfindahl-Hirschman Index (HHI), originally a market concentration measurement index, in the meta-features to replace IR(Imbalanced Ratio). Also, we developed a new index called Reverse ReLU Silhouette Score into the meta-feature set. Among the UCI Machine Learning Repository data, six representative datasets (Balance Scale, PageBlocks, Car Evaluation, User Knowledge-Modeling, Wine Quality(red), Contraceptive Method Choice) were selected. The class of each dataset was classified by using the classification algorithms (KNN, Logistic Regression, Nave Bayes, Random Forest, and SVM) selected in the study. For each dataset, we applied 10-fold cross validation method. 10% to 100% oversampling method is applied for each fold and meta-features of the dataset is measured. The meta-features selected are HHI, Number of Classes, Number of Features, Entropy, Reverse ReLU Silhouette Score, Nonlinearity of Linear Classifier, Hub Score. F1-score was selected as the dependent variable. As a result, the results of this study showed that the six meta-features including Reverse ReLU Silhouette Score and HHI proposed in this study have a significant effect on the classification performance. (1) The meta-features HHI proposed in this study was significant in the classification performance. (2) The number of variables has a significant effect on the classification performance, unlike the number of classes, but it has a positive effect. (3) The number of classes has a negative effect on the performance of classification. (4) Entropy has a significant effect on the performance of classification. (5) The Reverse ReLU Silhouette Score also significantly affects the classification performance at a significant level of 0.01. (6) The nonlinearity of linear classifiers has a significant negative effect on classification performance. In addition, the results of the analysis by the classification algorithms were also consistent. In the regression analysis by classification algorithm, Naïve Bayes algorithm does not have a significant effect on the number of variables unlike other classification algorithms. This study has two theoretical contributions: (1) two new meta-features (HHI, Reverse ReLU Silhouette score) was proved to be significant. (2) The effects of data characteristics on the performance of classification were investigated using meta-features. The practical contribution points (1) can be utilized in the development of classification algorithm recommendation system according to the characteristics of datasets. (2) Many data scientists are often testing by adjusting the parameters of the algorithm to find the optimal algorithm for the situation because the characteristics of the data are different. In this process, excessive waste of resources occurs due to hardware, cost, time, and manpower. This study is expected to be useful for machine learning, data mining researchers, practitioners, and machine learning-based system developers. The composition of this study consists of introduction, related research, research model, experiment, conclusion and discussion.

Halo Effect in Evaluating Government Funded Art Programs: The Case of Local Representative Performing Art Festivals (정부지원 공연예술행사 평가의 후광효과: 지역대표공연예술제 성과관리 체계를 중심으로)

  • Cho, Mun-Seok;Oh, Jae-Rok
    • Journal of Convergence for Information Technology
    • /
    • v.9 no.8
    • /
    • pp.123-133
    • /
    • 2019
  • This research empirically investigated halo effect in evaluating culture and art performance program. We diagnosed halo effect by using correlation analysis, factor analysis, and regression model on results and scores of fifteen evaluation indicators within three categories for the 107 Local Representative Performance Art Festivals in 2014 and 2015. The results indicates strong possibility of halo effect in culture and art performance evaluation. The correlation coefficients between evaluation indicators is higher than 0.5 and factor structure does not match with evaluation categories in both years. Scores in categories and standard deviations also are also significantly correlated with each other. The results implies that more sophisticated standard, diversification of evaluator, education, and meta-anlysis are need to control halo effect.

Post-diagnosis Soy Food Intake and Breast Cancer Survival: A Meta-analysis of Cohort Studies

  • Chi, Feng;Wu, Rong;Zeng, Yue-Can;Xing, Rui;Liu, Yang;Xu, Zhao-Guo
    • Asian Pacific Journal of Cancer Prevention
    • /
    • v.14 no.4
    • /
    • pp.2407-2412
    • /
    • 2013
  • Background and Objectives: Data on associations between soy food intake after cancer diagnosis with breast cancer survival are conflicting, so we conducted this meta-analysis for more accurate evaluation. Methods: Comprehensive searches were conducted to find cohort studies of the relationship between soy food intake after cancer diagnosis and breast cancer survival. Data were analyzed with comprehensive meta-analysis software. Results: Five cohort studies (11,206 patients) were included. Pooling all comparisons, soy food intake after diagnosis was associated with reduced mortality (HR 0.85, 95%CI 0.77 0.93) and recurrence (HR 0.79, 95%CI 0.72 0.87). Pooling the comparisons of highest vs. lowest dose, soy food intake after diagnosis was again associated with reduced mortality (HR 0.84, 95%CI 0.71 0.99) and recurrence (HR 0.74, 95%CI 0.64 0.85). Subgroup analysis of ER status showed that soy food intake was associated with reduced mortality in both ER negative (highest vs. lowest: HR 0.75, 95%CI 0.64 0.88) and ER positive patients (highest vs. lowest: HR 0.72, 95%CI 0.61 0.84), and both premenopausal (highest vs. lowest: HR 0.78, 95%CI 0.69 0.88) and postmenopausal patients (highest vs. lowest: HR 0.81, 95%CI 0.73 0.91). In additioin, soy food intake was associated with reduced recurrence in ER negative (highest vs. lowest: HR 0.64, 95%CI 0.44 0.94) and ER+/PR+ (highest vs. lowest: HR 0.65, 95%CI 0.49 0.86), and postmenopausal patients (highest vs. lowest: HR 0.67, 95%CI 0.56 0.80). Conclusion: Our meta-analysis showed that soy food intake might be associated with better survival, especially for ER negative, ER+/PR+, and postmenopausal patients.

Ki-67/MIB-1 as a Prognostic Marker in Cervical Cancer - a Systematic Review with Meta-Analysis

  • Piri, Reza;Ghaffari, Alireza;Gholami, Nasrin;Azami-Aghdash, Saber;PourAli-Akbar, Yasmin;Saleh, Parviz;Naghavi-Behzad, Mohammad
    • Asian Pacific Journal of Cancer Prevention
    • /
    • v.16 no.16
    • /
    • pp.6997-7002
    • /
    • 2015
  • Background: In cervical cancer patients it has been reported that there in a significant Ki-67/MIB-1 expression is correlated with survival in cervical cancer patients. However, the prognostic value is still not well understood. Materials and Methods: In the present meta-analysis the prognostic value of Ki-67/MIB-1 with regard to overall survival (OS) and disease-free survival (DFS) in cervical cancer was investigated. The databases of PubMed, ISI Web of Science, Cochrane Central Register of Controlled Trials, EMBASE, Science Direct and Wiley Online Library were used to identify appropriate literature. Results: In order to explore the relationship between Ki-67/MIB-1 and cervical cancer, we have included 13 studies covering 894 patients in the current meta-analysis. The effect of Ki-67/MIB-1 on OS for pooled random effects HR estimate was 1.63 (95%confidence interval (CI) 1.09-2.45; P<0.05). The pooled HR for DFS was 1.26 (95%CI 0.58-2.73; P>0.05) and the subgroup analysis indicated Ki-67/MIB1 was associated with DFS (HR=3.67, 95%CI 2.65-5.09) in Asians. Conclusions: According to this meta-analysis, Ki-67/MIB-1 has prognostic value for OS in patients suffering from cervical cancer. For better evaluation of the prognostic role of Ki-67/MIB-1 on DFS, studies with larger numbers of patients are needed to validate present findings in the future.

A Meta-analysis of the Association between Blood Lead and Blood Pressure (혈중 납과 혈압의 연관성에 관한 메타분석)

  • Koh, Sang-Baek;Nam, Chung-Mo;Choi, Hong-Ryul;Cha, Bong-Suk;Park, Jong-Ku;Jee, Ho-Sung;Kim, Chun-Bae
    • Journal of Preventive Medicine and Public Health
    • /
    • v.34 no.3
    • /
    • pp.262-268
    • /
    • 2001
  • Objectives : To integrate the results of studies which assess an association between blood lead and blood pressure. Methods : We surveyed the existing literature using a MEDLINE search with blood lead and blood pressure as key words, including reports published from January 1980 to December 2000. The criteria for quality evaluation were as follows: 1) the study subjects must have been workers exposed to lead, and 2) both blood pressure and blood lead must have been measured and presented with sufficient details so as to estimate or calculate the size of the association as a continuous variable. Among the 129 articles retrieved, 13 studies were selected for quantitative meta-analysis. Before the integration of each regression coefficient for the association between blood pressure and blood lead, a homogeneity test was conducted. Results : As the homogeneity of studies was rejected in a fixed effect model, we used the results in a random effect model. Our quantitative meta-analysis yielded weighted regression coefficients of blood lead associated with systolic blood pressure and diastolic blood pressure results of 0.0047 (95% confidence interval [CI]: -0.0061, 0.0155) and 0.0004 (95% CI: -0.0031, 0.0039), respectively. Conclusions : The published evidence suggested that there may be a weak positive association between blood lead and blood pressure, but the association is not significant.

  • PDF

Whole Brain Radiotherapy Plus Chemotherapy in the Treatment of Brain Metastases from Lung Cancer: A Meta-analysis of 19 Randomized Controlled Trails

  • Liu, Wen-Jing;Zeng, Xian-Tao;Qin, Hai-Feng;Gao, Hong-Jun;Bi, Wei-Jing;Liu, Xiao-Qing
    • Asian Pacific Journal of Cancer Prevention
    • /
    • v.13 no.7
    • /
    • pp.3253-3258
    • /
    • 2012
  • Objective: To evaluate the efficacy and safety of whole brain radiotherapy (WBRT) plus chemotherapy versus WBRT alone for treating brain metastases (BM) from lung cancer by performing a meta-analysis based on randomized controlled trials (RCTs). Methods: The PubMed, Embase, CENTRAL, ASCO, ESMO, CBM, CNKI, and VIP databases were searched for relevant RCTs performed between January 2000 and March 2012. After quality assessment and data extraction, the meta-analysis was performed using the RevMan 5.1 software, with funnel plot evaluation of publication bias. Results: 19 RCTs involving 1,343 patients were included. The meta-analyses demonstrated that compared to WBRT alone, WBRT plus chemotherapy was more effective with regard to the objective response rate (OR = 2.30, 95% CI = 1.79 - 2.98; P < 0.001); however, the incidences of gastrointestinal reactions (RR = 3.82, 95% CI = 2.33 - 6.28, P <0.001), bone marrow suppression (RR = 5.49, 95% CI = 3.65 - 8.25, P < 0.001), thrombocytopenia (RR = 5.83, 95% CI = 0.39 - 86.59; P = 0.20), leukopenia (RR = 3.13, 95% CI = 1.77 - 5.51; P < 0.001), and neutropenia (RR = 2.75, 95% CI = 1.61 - 4.68; P < 0.001) in patients treated with WBRT plus chemotherapy were higher than with WBRT alone. There was no obvious publication bias detected. Conclusion: WBRT plus chemotherapy can obviously improve total efficacy rate, butalso increases the incidence of adverse reactions compared to WBRT alone. From the limitations of this study, more large-scale, high-quality RCTs are suggested for further verification.

Meta-Analysis of Associations Between Classic Metric and Altmetric Indicators of Selected LIS Articles

  • Vysakh, C.;Babu, H. Rajendra
    • Journal of Information Science Theory and Practice
    • /
    • v.10 no.4
    • /
    • pp.53-65
    • /
    • 2022
  • Altmetrics or alternative metrics gauge the digital attention received by scientific outputs from the web, which is treated as a supplement to traditional citation metrics. In this study, we performed a meta-analysis of correlations between classic citation metrics and altmetrics indicators of library and information science (LIS) articles. We followed the systematic review method to select the articles and Erasmus Rotterdam Institute of Management Guidelines for reporting the meta-analysis results. To select the articles, keyword searches were conducted on Google Scholar, Scopus, and ResearchGate during the last week of November 2021. Eleven articles were assessed, and eight were subjected to meta-analysis following the inclusion and exclusion criteria. The findings reported negative and positive associations between citations and altmetric indicators among the selected articles, with varying correlation coefficient values from -.189 to 0.93. The result of the meta-analysis reported a pooled correlation coefficient of 0.47 (95% confidence interval, 0.339 to 0.586) for the articles. Sub-group analysis based on the citation source revealed that articles indexed on the Web of Science showed a higher pooled correlation coefficient (0.41) than articles indexed in Google Scholar (0.30). The study concluded that the pooled correlation between citation metrics with altmetric indicators was positive, ranging from low to moderate. The result of the study gives more insights to the scientometrics community to propose and use altmetric indicators as a proxy for traditional citation indicators for quick research impact evaluation of LIS articles.

Effectiveness of Electroacupuncture for Patients with Failed Back Surgery Syndrome: A Systematic Review and Meta-analysis

  • Shin, Donghoon;Shin, Kyungmoon;Jeong, Hwejoon;Kang, Deok;Yang, Jaewoo;Oh, Jihoon;Lim, Jinwoong
    • Journal of Acupuncture Research
    • /
    • v.39 no.3
    • /
    • pp.159-169
    • /
    • 2022
  • Failed back surgery syndrome (FBSS) is a term that applies to symptoms such as persistent or recurring low back pain, paresthesia, sciatica, or numbness after spine surgery. Electroacupuncture (EA) has been reported to have excellent analgesic effects although there have been no systematic reviews on the effects of EA on FBSS. Therefore, a systematic review and meta-analysis of the effectiveness of EA on FBSS was conducted. Eight databases were searched for studies that used EA for FBSS and 7 randomized controlled trials (RCTs) were included. RCTs of EA as combination therapy for FBSS compared with conventional treatment demonstrated improvement in the level of pain, lumbar functional scale scores, and quality of life. However, meta-analysis showed that reduction in pain was not statistically significant, while evaluation of lumbar function significantly improved, although the quality of evidence in the RCTs was generally low. RCTs comparing EA alone with conventional treatment demonstrated an improved level of pain, lumbar function, and effective rate of treatment. Meta-analysis showed that pain was significantly decreased in the EA alone group compared with the control group, although the quality of evidence was low. To improve the quality of evidence, high-quality RCTs are required in the future.