한국정보처리학회:학술대회논문집 (Proceedings of the Korea Information Processing Society Conference)
- 한국정보처리학회 2023년도 추계학술발표대회
- /
- Pages.628-631
- /
- 2023
- /
- 2005-0011(pISSN)
- /
- 2671-7298(eISSN)
DOI QR Code
Meme Analysis using Image Captioning Model and GPT-4
- Marvin John Ignacio (Dept. of Computer Engineering, Sejong University) ;
- Thanh Tin Nguyen (Dept. of Computer Engineering, Sejong University) ;
- Jia Wang (Dept. of Computer Engineering, Sejong University) ;
- Yong-Guk Kim (Dept. of Computer Engineering, Sejong University)
- 발행 : 2023.11.02
초록
We present a new approach to evaluate the generated texts by Large Language Models (LLMs) for meme classification. Analyzing an image with embedded texts, i.e. meme, is challenging, even for existing state-of-the-art computer vision models. By leveraging large image-to-text models, we can extract image descriptions that can be used in other tasks, such as classification. In our methodology, we first generate image captions using BLIP-2 models. Using these captions, we use GPT-4 to evaluate the relationship between the caption and the meme text. The results show that OPT6.7B provides a better rating than other LLMs, suggesting that the proposed method has a potential for meme classification.
키워드