MMAFFBen: A Multilingual and Multimodal Affective Analysis Benchmark for Evaluating LLMs and VLMs

This is an extensive open-source benchmark for multilingual multimodal affective analysis.

Paper link: MMAFFBen

Datasets

• MMAFFBen
• MMAFFIn

Model

• MMAFFLM-3b
• MMAFFLM-7b

Usage

Fine-tune your model based on MMAFFIn

Download the train datasets (MMAFFIn) to the data folder.

bash run_sft_stream.sh

Evaluate your model on MMAFFBen

Download MMAFFBen data to the data folder.

bash run_inference.sh

This code is based on LLaMA-Factory. The current version supports the Qwen-VL series. Adjust the code for your own model according to the guidelines according to LLaMA-Factory

After getting the predicted results in the predicts folder, follow the steps in the evaluation.ipynb to obtain the scores of each subdataset.

Citation

If you use MMAFFBen in your work, please cite our paper:

@article{liu2025mmaffben,
  title={MMAFFBen: A Multilingual and Multimodal Affective Analysis Benchmark for Evaluating LLMs and VLMs},
  author={Liu, Zhiwei and Qian, Lingfei and Xie, Qianqian and Huang, Jimin and Yang, Kailai and Ananiadou, Sophia},
  journal={arXiv preprint arXiv:2505.24423},
  year={2025}
}