README.md

Examing Bias and Fairness in Multimodal LLMs in Film History

Model

Open-weight Models

The models are listed by the format of 'Model Name: model name in command line'

• QwenVL-2.5: qwenvl2.5_7b
• InternVL-2.5
• LLaMA3.2-Vision
• Molmo
• ....

Private Models

• GPT-4o
• Claude
• Gemini-Pro

Inference

I collected the generation pipeline into one single file generate_response.py

python -m generate_response --model qwenvl2.5_7b

Data

TBD

Evaluation Pipeline

TBD