README.md

---
base_model: Qwen/Qwen2-VL-7B-Instruct
datasets:
- aliencaocao/multimodal_meme_classification_singapore
language: en
library_name: transformers
license: mit
metrics:
- accuracy
- roc_auc
pipeline_tag: image-text-to-text
tags:
- memes
- offensive
- singapore
- vlm
model-index:
- name: qwen2-vl-7b-rslora-offensive-meme-singapore
  results:
  - task:
      type: image-classification
      name: Offensive Meme Classification
    dataset:
      name: Offensive Memes in Singapore Context
      type: aliencaocao/multimodal_meme_classification_singapore
      split: test
    metrics:
    - type: roc_auc
      value: 0.8192
      name: AUROC
    - type: accuracy
      value: 0.8043
      name: Accuracy
---

# Model Card for Qwen2-VL 7B RSLORA Offensive Meme Singapore

This model is a fine-tuned version of Qwen2-VL-7B-Instruct for offensive meme classification in the Singapore context.  It was trained on the [multimodal_meme_classification_singapore](https://huggingface.co/datasets/aliencaocao/multimodal_meme_classification_singapore) dataset.

## Model Details

### Model Description

This model classifies memes as offensive or not, taking into account Singaporean social context. It leverages the visual and textual understanding capabilities of Qwen2-VL-7B-Instruct.

- **Developed by:** Cao Yuxuan, Wu Jiayang, Alistair Cheong Liang Chuen, Bryan Shan Guanrong, Theodore Lee Chong Jen, and Sherman Chann Zhi Shen
- **Model type:** Vision-Language Model (VLM)
- **Language(s) (NLP):** en
- **License:** MIT
- **Finetuned from model:** Qwen/Qwen2-VL-7B-Instruct

### Model Sources

- **Repository:** [https://github.com/aliencaocao/vlm-for-memes-aisg](https://github.com/aliencaocao/vlm-for-memes-aisg)
- **Paper:** [Detecting Offensive Memes with Social Biases in Singapore Context Using Multimodal Large Language Models](https://arxiv.org/abs/2502.18101)


## Uses

### Direct Use

This model can be used directly to classify memes.  See the code example in the "How to Get Started" section.

### Downstream Use [optional]

This model can be further fine-tuned for other related tasks or incorporated into a larger content moderation system.

### Out-of-Scope Use

This model is specifically trained for the Singaporean context and may not generalize well to other cultures or languages.  It should not be used to make definitive judgments about individuals or groups.

## Bias, Risks, and Limitations

Like any machine learning model, this model may exhibit biases present in the training data.  It is important to be aware of these limitations and use the model responsibly.  Further research is needed to assess and mitigate potential biases.

### Recommendations

Users should be aware of the potential for bias and limitations in the model's performance.  It is recommended to use this model as a tool to assist human moderators rather than a replacement for human judgment.

## How to Get Started with the Model

See the model repository's README for usage examples: [https://github.com/aliencaocao/vlm-for-memes-aisg](https://github.com/aliencaocao/vlm-for-memes-aisg)


## Training Details

### Training Data

The model was trained on the [multimodal_meme_classification_singapore](https://huggingface.co/datasets/aliencaocao/multimodal_meme_classification_singapore) dataset. This dataset contains memes labeled as offensive or not within the Singaporean context.

### Training Procedure

More details about the training procedure can be found in the paper.

## Evaluation

The model achieved an AUROC of 0.8192 and an accuracy of 0.8043 on a held-out test set.  See the paper for more details on the evaluation methodology.

## Citation

```bibtex
@misc{yuxuan2025detectingoffensivememessocial,
      title={Detecting Offensive Memes with Social Biases in Singapore Context Using Multimodal Large Language Models}, 
      author={Cao Yuxuan and Wu Jiayang and Alistair Cheong Liang Chuen and Bryan Shan Guanrong and Theodore Lee Chong Jen and Sherman Chann Zhi Shen},
      year={2025},
      eprint={2502.18101},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2502.18101}, 
}
```