International journal of advanced smart convergence

The Institute of Internet, Broadcasting and Communication (한국인터넷방송통신학회)
권호 표지
DOI QR코드

DOI QR Code

Application of ChatGPT text extraction model in analyzing rhetorical principles of COVID-19 pandemic information on a question-and-answer community

  • Hyunwoo Moon (Department of MetaBioHealth Sungkyunkwan University) ;
  • Beom Jun Bae (Department of Communication Arts Georgia Southern University Statesboro) ;
  • Sangwon Bae (Terry College of Business University of Georgia Athens)
  • Received : 2024.05.01
  • Accepted : 2024.05.26
  • Published : 2024.06.30
Download PDF
⟨ Previous Next ⟩

Abstract

This study uses a large language model (LLM) to identify Aristotle's rhetorical principles (ethos, pathos, and logos) in COVID-19 information on Naver Knowledge-iN, South Korea's leading question-and-answer community. The research analyzed the differences of these rhetorical elements in the most upvoted answers with random answers. A total of 193 answer pairs were randomly selected, with 135 pairs for training and 58 for testing. These answers were then coded in line with the rhetorical principles to refine GPT 3.5-based models. The models achieved F1 scores of .88 (ethos), .81 (pathos), and .69 (logos). Subsequent analysis of 128 new answer pairs revealed that logos, particularly factual information and logical reasoning, was more frequently used in the most upvoted answers than the random answers, whereas there were no differences in ethos and pathos between the answer groups. The results suggest that health information consumers value information including logos while ethos and pathos were not associated with consumers' preference for health information. By utilizing an LLM for the analysis of persuasive content, which has been typically conducted manually with much labor and time, this study not only demonstrates the feasibility of using an LLM for latent content but also contributes to expanding the horizon in the field of AI text extraction.

Keywords

References

  1. Yi, Y. J., "Sexual health information-seeking behavior on a social media site: Predictors of best answer selection," Online Information Review, Vol. 42, No. 6, pp. 880-897, Sep 2018.
  2. Jang, S. H., Jung, K. E., and Yi, Y. J., "The Power of Fake News: Big Data Analysis of Discourse About COVID-19-Related Fake News in South Korea," International Journal of Communication, Vol. 17, pp. 5527-5553, Aug. 2023.
  3. Tasnim, S., Hossain, M., and Mazumder, H., "Impact of rumors or misinformation on coronavirus disease (COVID-19) in social media," https://doi.org/10.31235/osf.io/uf3zn, 2020.
  4. Fang, X., & Wang, T. "Using natural language processing to identify effective influencers," International Journal of Market Research, Vol. 64, No. 5, pp. 611-629, 2022. DOI: https://doi.org/10.1177/14707853221101565.
  5. Luccioni, A. S., Pham, K. H., Lam, C. S. N., Aylett-Bullock, J., and Luengo-Oroz, M. "Ensuring the Inclusive Use of Natural Language Processing in the Global Response to COVID-19," arXiv:2108.10791, 2021. DOI: https://doi.org/10.48550/arXiv.2108.10791.
  6. Hasan, A., Levene, M., Weston, D., Fromson, R., Koslover, N., and Levene, T. "Monitoring COVID-19 on Social Media: Development of an End-to-End Natural Language Processing Pipeline Using a Novel Triage and Diagnosis Approach," Journal of Medical Internet Research, Vol. 23, No. 2, e30397, 2021. DOI: https://doi.org/10.2196/30397.
  7. Bae, B. J. and Yi, Y. J., "What answers do questioners want on social Q&A? User preferences of answers about STDs," Internet Research, Vol. 27, No. 5, pp. 1104-1121, Oct 2017.
  8. Hirvonen, N., Tirroniemi, A., and Kortelainen, T., "The cognitive authority of user-generated health information in an online forum for girls and young women," Journal of Documentation, Vol. 75, No. 1, pp. 78-98, Jan 2019.
  9. Zhao, W., Meng, K., Sun, L., Ma, J., and Jia, Z. "Language style and recognition of the answers in health Q&A community: Moderating effects of medical terminology," Journal of Information Science, Vol. 0, No. 0, May 2023. DOI: https://doi.org/10.1177/01655515231171367.
  10. Brown, A. D., Ainsworth, S., and Grant, D., "The rhetoric of institutional change," Organization Studies, Vol. 33, No. 3, pp. 297-321, Feb 2012.
  11. Iob, G., Visintini, C., and Palese, A. "Persuasive discourses in editorials published by the top-five nursing journals: Findings from a 5-year analysis," Nursing Philosophy, Vol. 23, No. 2, Dec 2022. DOI: https://doi.org/10.1111/nup.12378.
  12. Hall, I. J. and Johnson-Turbes, A., "Use of the persuasive health message framework in the development of a community-based mammography promotion campaign," Cancer Causes & Control, Vol. 26, pp. 775-784, Apr 2015.
  13. Brennan, N. M. and Merkl-Davies, D. M., "Rhetoric and argument in social and environmental reporting: the Dirty Laundry case," Accounting, Auditing & Accountability Journal, Vol. 27, No. 4, pp. 602-633, Apr 2014.
  14. Murphy, J. J., Rhetoric in the Middle Ages: A History of Rhetorical Theory from Saint Augustine to the Renaissance. Vol. 277, Berkeley, CA, USA: University of California Press, 1981.
  15. Yi, Y. J., Stvilia, B., and Mon, L., "Cultural influences on seeking quality health information: An exploratory study of the Korean community," Library & Information Science Research, Vol. 34, No. 1, pp. 45-51, Jan 2012.
  16. Buhi, E. R. et al., "Quality and accuracy of sexual health information web sites visited by young people," Journal of Adolescent Health, Vol. 47, No. 2, pp. 206-208, Aug 2010.
  17. Hanauska, M. and Lessmollmann, A., "Persuasion in Science Communication: empirical findings on scientific weblogs," Interaction Studies, Vol. 22, No. 3, pp. 343-372, Dec 2021.
  18. Rains, S. A. and Tukachinsky, R., "An examination of the relationships among uncertainty, appraisal, and information-seeking behavior proposed in uncertainty management theory," Health Communication, Vol. 30, No. 4, pp. 339-349, Jun 2014.
  19. Auger, G. A., "Rhetorical framing: Examining the message structure of nonprofit organizations on Twitter," International Journal of Nonprofit and Voluntary Sector Marketing, Vol. 19, No. 4, pp. 239-249, Sep 2014.
  20. Bronstein, J., "Like me! Analyzing the 2012 presidential candidates' Facebook pages," Online Information Review, Vol. 37, No. 2, pp. 173-192, Apr 2013.
  21. Iqbal, Z., Aslam, M. Z., Aslam, T., Ashraf, R., Kashif, M., and Nasir, H., "Persuasive power concerning COVID-19 employed by Premier Imran Khan: A socio-political discourse analysis," Register Journal, Vol. 13, No. 1, pp. 208-230, Jun 2020.
  22. Bos, L., Schemer, C., Corbu, N., Hameleers, M., Andreadis, I., Schulz, A., Schmuck, D., Reinemann, C., and Fawzi, N., "The effects of populism as a social identity frame on persuasion and mobilisation: Evidence from a 15-country experiment," European Journal of Political Research, Vol. 59, No. 1, pp. 3-24, May 2020.
  23. English, K., Sweetser, K. D., and Ancu, M., "YouTube-ification of political talk: An examination of persuasion appeals in viral video," American Behavioral Scientist, Vol. 55, No. 6, pp. 733-748, Mar 2011.
  24. Osborne, H., Health Literacy from A to Z: Practical Ways to Communicate Your Health Message. Lake Placid, NY: Aviva Publishing.