Journal of Preventive Medicine and Public Health

The Korean Society for Preventive Medicine (대한예방의학회)
권호 표지
DOI QR코드

DOI QR Code

Test-Retest Reliability of a Questionnaire for the Korea Youth Risk Behavior Web-Based Survey

  • Bae, Ji-Suk (Department of Preventive Medicine, Catholic University of Daegu School of Medicine) ;
  • Joung, Hyo-Jee (Seoul National University School of Public Health) ;
  • Kim, Jong-Yeon (Department of Preventive Medicine, Catholic University of Daegu School of Medicine) ;
  • Kwon, Kyoung-Nam (Lotte Samkang Co., Ltd.) ;
  • Kim, Young-Taek (Korea Centers for Disease Control and Prevention) ;
  • Park, Soon-Woo (Department of Preventive Medicine, Catholic University of Daegu School of Medicine)
  • Received : 2010.05.31
  • Accepted : 2010.08.04
  • Published : 2010.09.30
Download PDF
⟨ Previous Next ⟩

Abstract

Objectives: A web-based survey has been administered annually since 2005 throughout Korea to assess the prevalence of adolescent health risk behaviors among middle and high school students. The aim of this study was to evaluate the test-retest reliability of the Korea Youth Risk Behavior Web-based Survey (KYRBWS) questionnaire. Methods: A convenience sample of 2298 middle and high school students participated in self-administered questionnaire surveys twice, approximately two weeks apart, in 2008. The percent agreement, kappa statistics, and prevalence rates at the first and second surveys were computed for the core subset of 39 self-reported health risk behavior indices of the KYRBWS. Results: Among 39 indices, seven indices had kappas $\geq$ 0.81 and all of the indices had kappas $\geq$ 0.41. Based on nonoverlapping 95% confidence intervals, three indices had significantly different prevalence rates between the first and second surveys. In the subgroup analyses by school grade and gender, two indices had significantly different reliability estimates between middle and high school students. There were no significantly different reliability estimates between male and female students, except for one index. Conclusions: This study demonstrated that the reliability estimates for the KYRBWS questionnaire are varied, but generally reliable over time. The indices with low reliability estimates need to be evaluated further in order to determine whether the indices should be modified or deleted from future versions of the KYRBWS.

Keywords

References

  1. Korea National Statistical Office. Annual Report on the Cause of Death Statistics, 2007. Daejeon: Korea National Statistical Office; 2008. (Korean)
  2. Mokdad AH, Marks JS, Stroup DF, Gerberding JL. Actual causes of death in the United States, 2000. JAMA 2004; 291(10): 1238-1245. https://doi.org/10.1001/jama.291.10.1238
  3. McGue M, Iacono WG, Krueger R. The association of early adolescent problem behavior and adult psychopathology: a multivariate behavioral genetic perspective. Behav Genet 2006; 36(4): 591-602. https://doi.org/10.1007/s10519-006-9061-z
  4. Korea Centers for Disease Control and Prevention. Reports on the Korea Youth Risk Behavior Web-based Survey, 2005. Seoul: Korea Centers for Disease Control and Prevention; 2007. (Korean)
  5. Korea Centers for Disease Control and Prevention. Reports on the Korea Youth Risk Behavior Web-based Survey, 2006. Seoul: Korea Centers for Disease Control and Prevention; 2007. (Korean)
  6. Korea Centers for Disease Control and Prevention. Reports on the Korea Youth Risk Behavior Web-based Survey, 2007. Seoul: Korea Centers for Disease Control and Prevention; 2008. (Korean)
  7. Health Behaviour in School-aged Children. Health behaviour in school-aged children: a World Health Organization collaborative cross-national study. [cited 2010 May 31]. Available from: URL:http://www.hbsc.org.
  8. Eaton DK, Kann L, Kinchen S, Shanklin S, Ross J, Hawkins J, et al. Youth risk behavior surveillance--United States, 2007. MMWR Surveill Summ 2008; 57(4): 1-131.
  9. Brener ND, Collins JL, Kann L, Warren CW, Williams BI. Reliability of the Youth Risk Behavior Survey Questionnaire. Am J Epidemiol 1995; 141(6): 575-580. https://doi.org/10.1093/oxfordjournals.aje.a117473
  10. Brener ND, Kann L, McManus T, Kinchen SA, Sundberg EC, Ross JG. Reliability of the 1999 youth risk behavior survey questionnaire. J Adolesc Health 2002; 31(4): 336-342. https://doi.org/10.1016/S1054-139X(02)00339-7
  11. Zullig KJ, Pun S, Patton JM, Ubbes VA. Reliability of the 2005 middle school Youth Risk Behavior Survey. J Adolesc Health 2006; 39(6): 856-860. https://doi.org/10.1016/j.jadohealth.2006.07.008
  12. Vereecken CA, Maes L. A Belgian study on the reliability and relative validity of the Health Behaviour in School-Aged Children food-frequency questionnaire. Public Health Nutr 2003; 6(6): 581-588.
  13. Booth ML, Okely AD, Chey T, Bauman A. The reliability and validity of the physical activity questions in the WHO health behaviour in schoolchildren (HBSC) survey: a population study. Br J Sports Med 2001; 35(4): 263-267. https://doi.org/10.1136/bjsm.35.4.263
  14. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977; 33(1): 159-174. https://doi.org/10.2307/2529310
  15. Korean Educational Development Institute. Statistical Yearbook of Education, 2008. Seoul: Korean Educational Development Institute; 2008. (Korean)
  16. Feinstein AR, Cicchetti DV. High agreement but low kappa: I. The problems of two paradoxes. J Clin Epidemiol 1990; 43(6): 543-549. https://doi.org/10.1016/0895-4356(90)90158-L
  17. Cicchetti DV, Feinstein AR. High agreement but low kappa: II. Resolving the paradoxes. J Clin Epidemiol 1990; 43(6): 551-558. https://doi.org/10.1016/0895-4356(90)90159-M
  18. Byrt T, Bishop J, Carlin JB. Bias, prevalence and kappa. J Clin Epidemiol 1993; 46(5): 423-429. https://doi.org/10.1016/0895-4356(93)90018-V
  19. Brener ND, Billy JO, Grady WR. Assessment of factors affecting the validity of self-reported health-risk behavior among adolescents: evidence from the scientific literature. J Adolesc Health 2003; 33(6): 436-457. https://doi.org/10.1016/S1054-139X(03)00052-1
  20. Turner CF, Lessler JT, Devore JW. Effect of mode of administration and wording and reporting of drug use. In: Turner CF, Lessler JT, Gfroerer JC, editors. Survey Measurement of Drug Use: Methodological Studies. Washington, DC: Government Printing Office; 1992. p. 177-220.
  21. Turner CF, Ku L, Rogers SM, Lindberg LD, Pleck JH, Sonenstein FL. Adolescent sexual behavior, drug use, and violence: Increased reporting with computer survey technology. Science 1998; 280(5365): 867-873. https://doi.org/10.1126/science.280.5365.867
  22. Kann L, Brener ND, Warren CW, Collins JL, Giovino GA. An assessment of the effect of data collection setting on the prevalence of health-risk behaviors among adolescents. J Adolesc Health 2002; 31(4): 327-335. https://doi.org/10.1016/S1054-139X(02)00343-9

Cited by

  1. Association between physical activity and academic performance in Korean adolescent students vol.12, pp.1, 2012, https://doi.org/10.1186/1471-2458-12-258
  2. Association between weight status and recreational drug abuse in Korean adolescents vol.7, pp.2, 2010, https://doi.org/10.2478/s11536-011-0136-8
  3. The relationship between daily Internet use time and school performance in Korean adolescents vol.7, pp.4, 2010, https://doi.org/10.2478/s11536-012-0019-7
  4. Changes in Dietary Behavior Among Adolescents and Their Association With Government Nutrition Policies in Korea, 2005-2009 vol.45, pp.1, 2012, https://doi.org/10.3961/jpmph.2012.45.1.47
  5. Reliability and Validity of the Korean Version of Beck Depression Inventory-II via the Internet : Results from a University Student Sample vol.51, pp.6, 2012, https://doi.org/10.4306/jknpa.2012.51.6.402
  6. Effects of Objective and Subjective Socioeconomic Status on Self-Rated Health, Depressive Symptoms, and Suicidal Ideation in Adolescents vol.6, pp.3, 2013, https://doi.org/10.1007/s12187-013-9180-z
  7. Association between Overweight/Obesity and Academic Performance in South Korean Adolescents vol.21, pp.4, 2010, https://doi.org/10.21101/cejph.a3853
  8. Does Stress Increase the Risk of Atopic Dermatitis in Adolescents? Results of the Korea Youth Risk Behavior Web-Based Survey (KYRBWS-VI) vol.8, pp.8, 2010, https://doi.org/10.1371/journal.pone.0067890
  9. Does the Duration and Time of Sleep Increase the Risk of Allergic Rhinitis? Results of the 6-Year Nationwide Korea Youth Risk Behavior Web-Based Survey vol.8, pp.8, 2010, https://doi.org/10.1371/journal.pone.0072507
  10. 직장암 방사선치료기법별 자세오차에 관한 분석 vol.14, pp.12, 2010, https://doi.org/10.5762/kais.2013.14.12.6346
  11. Association between cigarette consumption and academic achievement in Korean adolescents vol.22, pp.5, 2014, https://doi.org/10.3109/16066359.2013.874417
  12. Objective and Subjective Socioeconomic Position and Current Smoking Among Korean Adolescents vol.15, pp.20, 2010, https://doi.org/10.7314/apjcp.2014.15.20.8877
  13. Physical Activity, Study Sitting Time, Leisure Sitting Time, and Sleep Time Are Differently Associated With Obesity in Korean Adolescents : A Population-Based Study vol.94, pp.44, 2010, https://doi.org/10.1097/md.0000000000001965
  14. Test-retest reliability of health behavior items in the Community Health Survey in South Korea vol.37, pp.None, 2010, https://doi.org/10.4178/epih/e2015045
  15. Association between Sleep Duration and Body Mass Index among South Korean Adolescents vol.15, pp.1, 2010, https://doi.org/10.15384/kjhp.2015.15.1.16
  16. Prevalence and Associated Factors of Subjective Halitosis in Korean Adolescents vol.10, pp.10, 2010, https://doi.org/10.1371/journal.pone.0140214
  17. Association Between Alcohol Consumption and Academic Achievement: a Cross-sectional Study vol.24, pp.1, 2010, https://doi.org/10.21101/cejph.a4292
  18. Happiness and health behaviors in South Korean adolescents: a cross-sectional study vol.38, pp.None, 2016, https://doi.org/10.4178/epih.e2016022
  19. Smoking, Physical Activity, and Eating Habits Among Adolescents vol.38, pp.1, 2010, https://doi.org/10.1177/0193945914544335
  20. Dietary Habits Are Associated With School Performance in Adolescents vol.95, pp.12, 2016, https://doi.org/10.1097/md.0000000000003096
  21. Factors Affecting Sexually Transmitted Infections in South Korean High School Students vol.33, pp.3, 2010, https://doi.org/10.1111/phn.12211
  22. Health Behaviors and Academic Performance Among Korean Adolescents vol.10, pp.2, 2010, https://doi.org/10.1016/j.anr.2016.01.004
  23. The Relationships Between Recreational Drug Abuse and School Records Among Korean Adolescents vol.26, pp.1, 2010, https://doi.org/10.1080/1067828x.2015.1009604
  24. Allergic rhinitis, atopic dermatitis, and asthma are associated with differences in school performance among Korean adolescents vol.12, pp.2, 2010, https://doi.org/10.1371/journal.pone.0171394
  25. The associations between internet use time and school performance among Korean adolescents differ according to the purpose of internet use vol.12, pp.4, 2010, https://doi.org/10.1371/journal.pone.0174878
  26. The Impact of Internet Addiction on Depression among Korean Adolescent Middle and High School Students vol.13, pp.2, 2010, https://doi.org/10.5392/ijoc.2017.13.2.006
  27. The association between electronic-cigarette use and self-reported oral symptoms including cracked or broken teeth and tongue and/or inside-cheek pain among adolescents: A cross-sectional study vol.12, pp.7, 2010, https://doi.org/10.1371/journal.pone.0180506
  28. Physical Activity, Sedentary Habits, Sleep, and Obesity are Associated with Asthma, Allergic Rhinitis, and Atopic Dermatitis in Korean Adolescents vol.58, pp.5, 2010, https://doi.org/10.3349/ymj.2017.58.5.1040
  29. Tapering Practices of Strongman Athletes: Test-Retest Reliability Study vol.6, pp.10, 2017, https://doi.org/10.2196/resprot.8522
  30. Atopic dermatitis is associated with active and passive cigarette smoking in adolescents vol.12, pp.11, 2010, https://doi.org/10.1371/journal.pone.0187453
  31. High stress, lack of sleep, low school performance, and suicide attempts are associated with high energy drink intake in adolescents vol.12, pp.11, 2010, https://doi.org/10.1371/journal.pone.0187759
  32. Active, passive, and electronic cigarette smoking is associated with asthma in adolescents vol.7, pp.None, 2010, https://doi.org/10.1038/s41598-017-17958-y
  33. Associations of multicultural status with depressive mood and suicidality among Korean adolescents: the roles of parental country of birth and socioeconomic position vol.17, pp.1, 2010, https://doi.org/10.1186/s12889-017-4044-y
  34. Lack of sleep is associated with internet use for leisure vol.13, pp.1, 2010, https://doi.org/10.1371/journal.pone.0191713
  35. Risk factors for disordered weight control behaviors among Korean adolescents: Multilevel analysis of the Korea Youth Risk Behavior Survey vol.51, pp.2, 2010, https://doi.org/10.1002/eat.22820
  36. Association between Parents' Country of Birth and Adolescent Depressive Symptoms: the Early Stages of Multicultural Society vol.33, pp.15, 2010, https://doi.org/10.3346/jkms.2018.33.e116
  37. Association between BMI for Obesity and Distress about Appearance in Korean Adolescents vol.33, pp.20, 2010, https://doi.org/10.3346/jkms.2018.33.e150
  38. Asthma and other allergic diseases in relation to suicidal behavior among South Korean adolescents vol.115, pp.None, 2010, https://doi.org/10.1016/j.jpsychores.2018.10.015
  39. Development of a suicide index model in general adolescents using the South Korea 2012–2016 national representative survey data vol.9, pp.None, 2010, https://doi.org/10.1038/s41598-019-38886-z
  40. Contextual Association between Political Regime and Adolescent Suicide Risk in Korea: A 12-year Repeated Cross-Sectional Study from Korea vol.16, pp.5, 2010, https://doi.org/10.3390/ijerph16050874
  41. Social inclusion among mental health service users: A reliability test of the Mini‐SCOPE Scale vol.28, pp.9, 2010, https://doi.org/10.1111/jocn.14798
  42. Prediction models for high risk of suicide in Korean adolescents using machine learning techniques vol.14, pp.6, 2019, https://doi.org/10.1371/journal.pone.0217639
  43. Association of alcohol and drug use with use of electronic cigarettes and heat-not-burn tobacco products among Korean adolescents vol.14, pp.7, 2010, https://doi.org/10.1371/journal.pone.0220241
  44. Test-retest repeatability of questionnaire for pain symptoms for school children aged 10-15 years vol.19, pp.3, 2019, https://doi.org/10.1515/sjpain-2018-0338
  45. Associations between Harmful Experiences from Alcohol Use of Others and Mental Health in Korean Adolescents vol.16, pp.21, 2010, https://doi.org/10.3390/ijerph16214240
  46. Association Between Weight Misperception and Some Mental Health-Related Characteristics in Korean Adolescents vol.16, pp.None, 2010, https://doi.org/10.2147/ndt.s286470
  47. Violence Victimization Among Adolescents from Culturally or Ethnically Diverse South Korean Families vol.22, pp.2, 2010, https://doi.org/10.1007/s10903-019-00957-8
  48. The association between high hygiene scores and allergic rhinitis in Korean adolescents vol.10, pp.8, 2010, https://doi.org/10.1002/alr.22569
  49. Associations Between Secondhand Smoke and Mental Health in Korean Adolescents vol.32, pp.8, 2010, https://doi.org/10.1177/1010539520957846
  50. Identification of oral symptoms associated with atopic dermatitis in adolescents: Results from the Korea national representative survey 2009–2017 vol.10, pp.1, 2020, https://doi.org/10.1038/s41598-020-76532-1
  51. The Association between Vigorous Physical Activity and Stress in Adolescents with Asthma vol.18, pp.7, 2010, https://doi.org/10.3390/ijerph18073467
  52. Energy-drink consumption is associated with asthma, allergic rhinitis, and atopic dermatitis in Korean adolescents vol.75, pp.7, 2010, https://doi.org/10.1038/s41430-020-00812-2
  53. A Study on the Perception Change of Passengers on Sustainable Air Transport Following COVID-19 Progress vol.13, pp.14, 2010, https://doi.org/10.3390/su13148056
  54. Asthma, Allergic Rhinitis, and Atopic Dermatitis Incidence in Korean Adolescents before and after COVID-19 vol.10, pp.15, 2010, https://doi.org/10.3390/jcm10153446
  55. Changes in Dietary Habits and Exercise Pattern of Korean Adolescents from Prior to during the COVID-19 Pandemic vol.13, pp.10, 2010, https://doi.org/10.3390/nu13103314