pytorch m2m_100 text2text-generation ca zh dataset:projecte-aina/ca_zh_wikipedia transformers license:cc-by-4.0 model-index autotrain_compatible
---
license: cc-by-4.0
language:
- ca
- zh
datasets:
- projecte-aina/ca_zh_wikipedia
metrics:
- "bleu"
model-index:
- name: m2m100_418M_ft_zh_ca 
  results:
  - task: 
      type: translation 
    dataset:
      type: flores
      name: Flores
    metrics:
      - name: BLEU
        type: bleu
        value: 18.0
---
## m2m100 fine-tuned on the ca_zh_wikipedia dataset for machine translation

## Table of Contents
- [Model Description](#model-description)
- [Intended Uses and Limitations](#intended-use)
- [How to Use](#how-to-use)
- [Training](#training)
  - [Training Data](#training-data)
  - [Training Procedure](#training-procedure)
    - [Tokenization](#tokenization)
    - [Hyperparameters](#hyperparameters)
- [Evaluation](#evaluation)
   - [Variable and Metrics](#variable-and-metrics)
   - [Evaluation Results](#evaluation-results)
- [Licensing Information](#licensing-information)
- [Citation Information](#citation-information)
- [Funding](#funding)

## Model description

This model was obtained by fine-tuning the [m2m100_418M](https://huggingface.co/facebook/m2m100_418M) model on a Zh-Ca machine translation task with the [ca_zh_wikipedia](https://huggingface.co/datasets/projecte-aina/ca_zh_wikipedia) dataset that has been created along with the model. We also evaluate it on a general-domain multilingual testset [Flores-101](https://github.com/facebookresearch/flores).

## Intended Uses and Limitations

You can use this model for machine translation from Chinese to Catalan. 

## How to use

```python
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

tokenizer = AutoTokenizer.from_pretrained("projecte-aina/m2m100_418M_ft_zh_ca")

model = AutoModelForSeq2SeqLM.from_pretrained("projecte-aina/m2m100_418M_ft_zh_ca")
```

## Training

### Training Data

As a data for fine-tuning we used the [ca_zh_wikipedia](https://huggingface.co/datasets/projecte-aina/ca_zh_wikipedia) dataset extracted from Wikipedia.

### Training Procedure

#### Tokenization

The original [m2m100_418M](https://huggingface.co/facebook/m2m100_418M) model's sentencepiece tokenizer was used. The fine-tuning dataset that contained both simplified and traditional Chinese was reduced to its simplified form.

#### Hyperparameters

The model was trained for 15 epochs with the default parameters and \\(LR = 2\mathrm{e}{-5}\\).

## Evaluation

### Variable and Metrics

We use the BLEU score for evaluation on test sets: [Flores-101](https://github.com/facebookresearch/flores).

### Evaluation Results

Below are the evaluation results on the machine translation from Catalan to Chinese compared with the original m2m100 on a testset: [Flores-101](https://github.com/facebookresearch/flores).

|Test set  | Model  | BLEU  |
| ------------|-------------| -----|
|Flores-101 | m2m100  |  17.5  | 
| | m2m100_418M_ft_ca_zh  |  **18.0**  |  

## Licensing Information

[Apache License, Version 2.0](https://www.apache.org/licenses/LICENSE-2.0)

## Funding

This work was funded by the [Departament de la Vicepresidència i de Polítiques Digitals i Territori de la Generalitat de Catalunya](https://politiquesdigitals.gencat.cat/ca/inici/index.html#googtrans(ca|en) within the framework of [Projecte AINA](https://politiquesdigitals.gencat.cat/ca/economia/catalonia-ai/aina).
