Skip to content

Latest commit

 

History

History
82 lines (57 loc) · 3.69 KB

README.md

File metadata and controls

82 lines (57 loc) · 3.69 KB

MemeCLIP: Leveraging CLIP Representations for Multimodal Meme Classification

Siddhant Bikram Shah, Shuvam Shiwakoti, Maheep Chaudhary, Haohan Wang

This is the code repository for our paper MemeCLIP: Leveraging CLIP Representations for Multimodal Meme Classification published in EMNLP 2024.

PrideMM Dataset

The images and labels for the PrideMM dataset are available here (Warning: Insensitive content).

Annotation Terminology

Hate

Class Terminology
No Hate 0
Hate 1

Targets of Hate

Class Terminology
Undirected 0
Individual 1
Community 2
Organization 3

Stance

Class Terminology
Neutral 0
Support 1
Oppose 2

Humor

Class Terminology
No Humor 0
Humor 1

MemeCLIP Code

All experimental changes can be made through a single file: configs.py.

Directory names can be set in the following variables:

  • cfg.root_dir
  • cfg.img_folder
  • cfg.info_file
  • cfg.checkpoint_path
  • cfg.checkpoint_file

To train, validate, and test MemeCLIP, set cfg.test_only = False and run main.py.

To test MemeCLIP, set cfg.test_only = True and run main.py.

CSV files are expected to contain image path, text, and label in no particular order.

Pre-trained Weights

Pre-trained weights for MemeCLIP (Hate Classification Task) are available here.

Citation

@article{shah2024memeclip,
  title={MemeCLIP: Leveraging CLIP Representations for Multimodal Meme Classification},
  author={Shah, Siddhant Bikram and Shiwakoti, Shuvam and Chaudhary, Maheep and Wang, Haohan},
  journal={arXiv preprint arXiv:2409.14703},
  year={2024}
}

OR

Siddhant Bikram Shah, Shuvam Shiwakoti, Maheep Chaudhary, and Haohan Wang. 2024. MemeCLIP: Leveraging CLIP Representations for Multimodal Meme Classification. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 17320–17332, Miami, Florida, USA. Association for Computational Linguistics.