Journal of Practical Engineering Education (실천공학교육논문지)

Korean Institute for Pratical Engineering Education (한국실천공학교육학회)
권호 표지
DOI QR코드

DOI QR Code

An Alternative Approach to Securing Research Reproducibility : Researcher Experience and Utilization of Statistical Software

연구 재현성 확보를 위한 대안적 접근 : 연구자 경험과 통계 소프트웨어 활용

  • Su-Hyun Ahn (College of General Education, Semyung University) ;
  • Sang-Jun Lee (College of General Education, Semyung University)
  • 안수현 (세명대학교 교양대학) ;
  • 이상준 (세명대학교 교양대학)
  • Received : 2024.10.12
  • Accepted : 2024.10.22
  • Published : 2024.10.31
Download PDF
⟨ Previous Next ⟩

Abstract

The purpose of this study is to explore the possibility of using open-source statistical software Jamovi as an alternative approach to securing reproducibility. To this end, a descriptive survey was conducted on researchers in various academic fields to investigate the causes of the reproducibility problem and its solutions, and whether the use of Jamovi could contribute to solving the statistical reproducibility problem was empirically analyzed. The function of integrating Jamovi's data and analysis results into a single file played an important role in ensuring transparency in analysis. In addition, the possibility of enhancing statistical reproducibility was confirmed by using Jamovi's Rj Editor module to convert all procedures in the analysis process into R codes while performing advanced statistical analysis. This study emphasizes that Jamovi can act as a practical tool for solving research reproducibility problems, and through this, it is expected to contribute to strengthening the reliability of academic research.

본 연구는 재현성 확보를 위한 대안적 접근으로 오픈소스 통계 소프트웨어 Jamovi의 활용 가능성을 탐색하는데 목적이 있다. 이를 위해 다양한 학문 분야의 연구자들을 대상으로 서술형 설문조사를 실시하여 재현성 문제의 원인과 그 해결 방안을 조사하였고, Jamovi의 사용이 통계적 재현성 문제 해결에 기여할 수 있는지에 대해 실증적으로 분석하였다. 연구 결과 Jamovi의 데이터와 분석 결과를 하나의 파일로 통합하는 기능은 분석의 투명성을 보장하는데 중요한 역할을 하였다. 또한 Jamovi의 Rj Editor 모듈을 활용하면 고급 통계 분석을 수행하면서 분석 과정의 모든 절차를 R 코드로 변환할 수 있어 통계적 재현성을 높일 수 있는 가능성을 확인하였다. 본 연구는 Jamovi가 연구 재현성 문제 해결을 위한 실질적인 도구로 작용할 수 있음을 강조하며, 이를 통해 학문적 연구의 신뢰성을 강화하는 데 기여할 수 있을 것으로 기대한다.

Keywords

Acknowledgement

본 과제(결과물)는 2024년도 교육부의 재원으로 한국연구재단의 지원을 받아 수행된 지자체-대학 협력기반 지역혁신사업의 결과입니다(2021RIS-001(1345370811)).

References

  1. J. T. Cacioppo, R. M. Kaplan, J. A. Krosnick, J. L. Olds, and H. Dean, Social, Behavioral, and Economic Sciences Perspectives on Robust and Reliable Science. Report of the Subcommittee on Replicability in Science Advisory Committee to the National Science Foundation Directorate for Social, Behavioral, and Economic Sciences, 1, 2015.
  2. S. N. Goodman, D. Fanelli, and J. P. Ioannidis, "What does research reproducibility mean?," Science Translational Medicine, vol. 8, no. 341, pp. 341ps12-341ps12, 2016.
  3. https://www.edge.org/response-detail/25340.
  4. M. Baker, "1,500 scientists lift the lid on reproducibility," Nature, vol. 533, no. 7604, pp. 452-454, 2016.
  5. C. G. Begley and J. P. Ioannidis, "Reproducibility in science: improving the standard for basic and preclinical research," Circulation Research, vol. 116, no. 1, pp. 116-126, 2015.
  6. J. B. Asendorpf, M. Conner, F. De Fruyt, J. De Houwer, J. J. Denissen, K. Fiedler, D. C. Funder, R. Kliegl, B. A. Nosek, M. Perugini, B. W. Roberts, M. Schmitt, M. A. G. Van Aken, H. Weber, and J. M. Wicherts, "Recommendations for increasing replicability in psychology," European Journal of Personality, vol. 27, no. 2, pp. 108-119, 2013.
  7. J. P. Simmons, L. D. Nelson, and U. Simonsohn, "Falsepositive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant," Psychological Science, vol. 22, no. 11, pp. 1359-1366, 2011.
  8. T. Yarkoni and J. Westfall, "Choosing prediction over explanation in psychology: Lessons from machine learning," Perspectives on Psychological Science, vol. 12, no. 6, pp. 1100-1122, 2017.
  9. M. C. Makel, J. A. Plucker, and B. Hegarty, "Replications in psychology research: How often do they really occur?," Perspectives on Psychological Science, vol. 7, no. 6, pp. 537-542, 2012.
  10. H. Pashler and E. J. Wagenmakers, "Editors' introduction to the special section on replicability in psychological science: A crisis of confidence?," Perspectives on Psychological Science, vol. 7, no. 6, pp. 528-530, 2012.
  11. T. Yarkoni and J. Westfall, "Choosing prediction over explanation in psychology: Lessons from machine learning," Perspectives on Psychological Science, vol. 12, no. 6, pp. 1100-1122, 2017.
  12. J. T. Leek and R. D. Peng, "Reproducible research can still be wrong: adopting a prevention approach," Proceedings of the National Academy of Sciences, vol. 112, no. 6, pp. 1645-1646, 2015.
  13. R. D. Peng, "Reproducible research in computational science," Science, vol. 334, no. 6060, pp. 1226-1227, 2011.
  14. M. D. Wilkinson, The FAIR Guiding Principle for Scientific Data Management and Stewardship: Comment, 2016.
  15. B. A. Nosek, G. Alter, G. C. Banks, D. Borsboom, S. D. Bowman, S. J. Breckler, ... M. Humphreys, and T. Yarkoni, "Promoting an open research culture," Science, vol. 348, no. 6242, pp. 1422-1425, 2015.
  16. M. Sahin and E. Aybek, "Jamovi: an easy to use statistical software for the social scientists," International Journal of Assessment Tools in Education, vol. 6, no. 4, pp. 670-692, 2019.
  17. G. Guest, A. Bunce, and L. Johnson, "How many interviews are enough? An experiment with data saturation and variability," Field Methods, vol. 18, no. 1, pp. 59-82, 2006.
  18. M. Q. Patton, Qualitative research & evaluation methods: Integrating theory and practice. Sage Publications, 2014.
  19. J. W. Creswell and C. N. Poth, Qualitative inquiry and research design: Choosing among five approaches. Sage Publications, 2016.
  20. A. Bryman, Social Research Methods. Oxford University Press, 2016.
  21. R. A. Krueger, Focus Groups: A Practical Guide for Applied Research. Sage Publications, 2014.
  22. The jamovi project, jamovi. (Version 2.4) [Computer Software]. Retrieved from https://www.jamovi.org, 2023.
  23. R Core Team, R: A Language and environment for statistical computing. (Version 4.1) [Computer software]. Retrieved from https://cran.r-project.org. (R packages retrieved from CRAN snapshot 2023-04-07), 2022.
  24. https://docs.jamovi.org/_pages/Rj_overview.html.