Browse > Article
http://dx.doi.org/10.5933/JKAPD.2013.40.3.223

Review on Problems with Null Hypothesis Significance Testing in Dental Research and Its Alternatives  

Lee, Kwang-Hee (Department of Pediatric Dentistry, College of Dentistry, Wonkwang University)
Publication Information
Journal of the korean academy of Pediatric Dentistry / v.40, no.3, 2013 , pp. 223-232 More about this Journal
Abstract
There are many problems in evaluating study results by p value in null hypothesis testing for dental research. It is a logical fallacy to conclude that the null hypothesis is true when the it is not rejected. There are much serious misunderstanding about p value, and researchers should be cautious about interpreting p value in writing papers. As alternatives to complement or replace the null hypothesis significance testing, effect size, confidence interval, and Bayesian statistics are introduced.
Keywords
Null hypothesis; Significance testing; p value; Effect size; Confidence interval; Bayesian statistics;
Citations & Related Records
연도 인용수 순위
  • Reference
1 Denis DJ : Alternatives to null hypothesis significance testing. Theory & Science, 4(1), 2003. Available from URL: http://theoryandscience.icaap.org/content/vol4.1/02_denis.html (Accessed on July 8, 2013)
2 Rosenthal R : Effect size estimation, significance testing, and the file-drawer problem. J Parapsychol, 56:57-58, 1992.
3 Vaughan GM, Corballis MC : Beyond tests of significance: Estimating strength of effects in selected ANOVA designs. Psychol Bulletin, 72:204-213, 1969.   DOI
4 Silva-Aycaguer LC, Suarez-Gil P, Fernandez-Somoano A : Null hypothesis significance test in health sciences research (1995-2006): statistical analysis and interpretation. BMC Med Res Methodol, 10:44, 2010.   DOI   ScienceOn
5 Schmidt FL : Statistical significance testing and cumulative knowledge in psychology: implications for training of researchers. Psychol Methods, 1:115-129, 1996.   DOI
6 Cumming G, Finch S : Inference by eye: confidence intervals and how to read pictures of data. Am Psychol, 60:170-180, 2005.   DOI   ScienceOn
7 Schenker N, Gentleman JF : On judging the significance of differences by examining the overlap between confidence intervals. Am Statistician, 55:182-186, 2001.   DOI   ScienceOn
8 Wang S, Campbell B : Mr. Bayes goes to Washington. Science, 339:758-759, 2013.   DOI
9 Efron B : Bayes’Theorem in the twenty-first century. Science, 340:1177-1178, 2013.   DOI   ScienceOn
10 FDA : Guidance for the use of Bayesian statistics in medical device clinical trials. Available from URL : http://www.fda.gov/medicaldevices/deviceregulationandguidance/guidancedocuments/ucm071072.htm (Accessed on July 8, 2013)
11 Carver RP : The case against statistical significance testing. Harvard Educat Review, 48:378-399, 1978.   DOI
12 Nickerson RS : Null hypothesis statistical testing: a review of an old and continuing controversy. Psychol Methods, 5:241-301, 2000.   DOI   ScienceOn
13 Berger JO, Sellke T : Testing a point null hypothesis: the irreconcilability of p values and evidence (with comments). J Am Stat Assoc, 82:112-139, 1987.
14 Berger JO, Delampady M : Testing precise hypotheses (with comments). Stat Science, 2:317-352, 1987.   DOI   ScienceOn
15 Nester MR : An applied statistician’s creed. Statistician, 45:401-410, 1996.
16 Berger JO, Berry DA : Statistical analysis and the illusion of objectivity. Am Scientist, 76:159-165, 1988.
17 Hubbard, R : Alphabet soup: blurring the distinctions between p's and ${\alpha}$'s in psychological research. Theory Psychol, 14:295-327, 2004.   DOI
18 Sellke T, Bayarri MJ, Berger JO : Calibration of p values for testing precise null hypotheses. Am Statistician, 55:62-71, 2001.   DOI   ScienceOn
19 Schervish MJ : P values: what they are and what they are not. Am Stat, 50:203-206, 1996.
20 Gelman A, Stern H : The difference between 'significant' and 'not significant' is not itself statistically significant. Am Statistician, 60:328-331, 2006.   DOI   ScienceOn
21 International committee of medical journal editors : Uniform requirements for manuscripts submitted to biomedical journals. Available from URL: http://www.icmje.org/manuscript_1prepare.html (Assessed on June 27, 2013)
22 Royall RM : The effect of sample size on the meaning of significance tests. Am Statistician, 40:313-315, 1986.
23 Hand DJ : Data mining: statistics and more? Am Statistician, 52:112.118, 1998.
24 Schmidt FL, Hunter JE : Eight common but false objections to the discontinuation of significance testing in the analysis of research data. In Harlow LA, Mulaik SA, Steiger JH (eds.) : What if there were no significance tests? Mahwah, NJ, Lawrence Erlbaum Associates, 37-64, 1997.
25 Meehl PE : Theory-testing in psychology and physics: a methodological paradox. Philosophy Sci, 34:103-115, 1967.   DOI   ScienceOn
26 Meehl PE : Theoretical risks and tabular asterisks: sir Karl, sir Ronald, and the slow progress of soft psychology. J Consult Clin Psychol, 46:806-834, 1978.   DOI
27 Cohen J : The earth is round (p<.05). Am Psychol, 49:997-1003, 1994.   DOI   ScienceOn
28 NHST problems. Available from URL: http://www.faculty.biol.ttu.edu/strauss/stats/LectureNotes/20_NHSTProblems.pdf (Accessed on July 8, 2013)
29 Fallacy of affirming the consequent. Available from URL: http://terms.naver.com/entry.nhn?cid=1137&docId=275047&mobile&categoryId=1137 (Accessed on July 8, 2013)
30 Pollard P, Richardson JTE : On the probability of making type I errors. Psychol Bull, 102:159-163, 1987.   DOI
31 Reese HW : Problems of statistical inference. Mex J Behav Anal, 25:39-68, 1999.
32 Goodman S : A dirty dozen: twelve p-value misconceptions. Semin Hematol, 45:135-140, 2008.   DOI   ScienceOn
33 Hubbard R, Lindsay RM : Why p values are not a useful measure of evidence in statistical significance sesting. Theory Psychol, 18:69-88, 2008.   DOI
34 Sterne JAC, Smith GD : Sifting the evidence - what's wrong with significance tests? BMJ(Clin res), 322:226-231, 2001.
35 Johnson, DH : The insignificance of statistical significance testing. J Wildlife Manag, 63:763-772, 1999.   DOI
36 Nurminen M, Mutanen P : Exact Bayesian analysis of two proportions. Scand J Stat, 14:67-77, 1987.
37 Nurminen M : Statistical significance - a misconstrued notion in medical research. Scand J Work Environ Health, 23:232-235, 1997.   DOI
38 Seaman JE, Allen IE : Not significant, but Important? Know the pitfalls of p-values and formal hypothesis tests. Quality Progress, 2011 August. Available from URL : http://asq.org/quality-progress/2011/08/statistics-roundtable/not-significant-butimportant.html (Accessed on July 8, 2013)
39 Matrixx Initiatives, Inc. v. Siracusano. Available from URL: http://en.wikipedia.org/wiki/Matrixx_Initiatives,_Inc._v._Siracusano (Accessed on July 8, 2013)
40 Efron B : Why isn’t everyone a Bayesian (with discussion)? Am Statist, 40:1-11, 1986.
41 Diaconis P, Freedman D : On the consistency of Bayes estimate (with discussion). Ann Math Stat, 14:1-67, 1986.   DOI
42 Freedman L : Bayesian statistical methods. A natural way to assess clinical evidence (editorial). Br Med J, 313:569-570, 1996.   DOI
43 Zhang Y, Todem D, Kim K, Lesaffre E : Bayesian latent variable models for spatially correlated toothlevel binary data in caries research. Stat Modelling, 11:25-47, 2011.   DOI
44 Tu YK, Needleman I, Chambrone L, et al. : A Bayesian network meta-analysis on comparisons of enamel matrix derivatives, guided tissue regeneration and their combination therapies. J Clin Periodontol, 39:303-314, 2012.   DOI   ScienceOn
45 Frosio I, Olivieri C, Lucchese M, et al. : Bayesian denoising in digital radiography: a comparison in the dental field. Comput Med Imaging Graph, 37:28-39, 2013.   DOI   ScienceOn
46 Lilford RJ, Braunholtz D : The statistical basis of public policy: a paradigm shift is overdue. Br Med J, 313:603-607, 1996.   DOI
47 Fisher RA : The design of experiments (8th ed.). Edinburgh, Oliver & Boyd, 1966.
48 Fisher BJ : R.A. Fisher: The life of a scientist. New York, Wiley, 1978.