pytorch marian text2text-generation transformers generated_from_trainer license:apache-2.0 model-index autotrain_compatible
---
license: apache-2.0
tags:
- generated_from_trainer
model-index:
- name: MIX1_ja-en_helsinki
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# MIX1_ja-en_helsinki

This model is a fine-tuned version of [Helsinki-NLP/opus-mt-ja-en](https://huggingface.co/Helsinki-NLP/opus-mt-ja-en) on a combination of Visual Novel, Light Novel, and Subtitle data. A total of ~10MM lines of training data were used.
It achieves the following results on the evaluation set:
- Loss: 1.7947
- Otaku Benchmark VN BLEU: 17.78
- Otaku Benchmark LN BLEU: 11.80
- Otaku Benchmark MANGA BLEU: 13.66

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0003
- train_batch_size: 64
- eval_batch_size: 64
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 2
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch | Step   | Validation Loss |
|:-------------:|:-----:|:------:|:---------------:|
| 2.7495        | 0.01  | 2000   | 2.5989          |
| 2.5415        | 0.03  | 4000   | 2.4746          |
| 2.4409        | 0.04  | 6000   | 2.4731          |
| 2.3743        | 0.05  | 8000   | 2.4012          |
| 2.3254        | 0.06  | 10000  | 2.3904          |
| 2.2857        | 0.08  | 12000  | 2.3649          |
| 2.2448        | 0.09  | 14000  | 2.3188          |
| 2.2158        | 0.1   | 16000  | 2.2975          |
| 2.193         | 0.11  | 18000  | 2.2756          |
| 2.1669        | 0.13  | 20000  | 2.2852          |
| 2.144         | 0.14  | 22000  | 2.2689          |
| 2.1222        | 0.15  | 24000  | 2.2721          |
| 2.1045        | 0.16  | 26000  | 2.2489          |
| 2.0885        | 0.18  | 28000  | 2.2359          |
| 2.0732        | 0.19  | 30000  | 2.2771          |
| 2.0584        | 0.2   | 32000  | 2.2582          |
| 2.0471        | 0.21  | 34000  | 2.2093          |
| 2.0369        | 0.23  | 36000  | 2.1768          |
| 2.0241        | 0.24  | 38000  | 2.1884          |
| 2.0196        | 0.25  | 40000  | 2.2025          |
| 2.004         | 0.27  | 42000  | 2.1507          |
| 1.9936        | 0.28  | 44000  | 2.1668          |
| 1.9869        | 0.29  | 46000  | 2.1432          |
| 1.9735        | 0.3   | 48000  | 2.1662          |
| 1.9651        | 0.32  | 50000  | 2.1824          |
| 1.9551        | 0.33  | 52000  | 2.1608          |
| 1.9485        | 0.34  | 54000  | 2.1322          |
| 1.9421        | 0.35  | 56000  | 2.1476          |
| 1.9303        | 0.37  | 58000  | 2.0994          |
| 1.9236        | 0.38  | 60000  | 2.1182          |
| 1.9183        | 0.39  | 62000  | 2.1305          |
| 1.9108        | 0.4   | 64000  | 2.1469          |
| 1.9051        | 0.42  | 66000  | 2.1414          |
| 1.9018        | 0.43  | 68000  | 2.1089          |
| 1.8959        | 0.44  | 70000  | 2.0908          |
| 1.886         | 0.46  | 72000  | 2.0968          |
| 1.8802        | 0.47  | 74000  | 2.0503          |
| 1.8713        | 0.48  | 76000  | 2.0542          |
| 1.8648        | 0.49  | 78000  | 2.0990          |
| 1.8599        | 0.51  | 80000  | 2.1112          |
| 1.8563        | 0.52  | 82000  | 2.1007          |
| 1.8541        | 0.53  | 84000  | 2.0849          |
| 1.845         | 0.54  | 86000  | 2.0831          |
| 1.8448        | 0.56  | 88000  | 2.0560          |
| 1.8342        | 0.57  | 90000  | 2.0349          |
| 1.8344        | 0.58  | 92000  | 2.0301          |
| 1.8291        | 0.59  | 94000  | 2.0300          |
| 1.819         | 0.61  | 96000  | 2.0378          |
| 1.8154        | 0.62  | 98000  | 2.0197          |
| 1.82          | 0.63  | 100000 | 2.0463          |
| 1.8081        | 0.64  | 102000 | 2.0077          |
| 1.8046        | 0.66  | 104000 | 2.0101          |
| 1.7978        | 0.67  | 106000 | 2.0150          |
| 1.7934        | 0.68  | 108000 | 2.0215          |
| 1.7904        | 0.7   | 110000 | 2.0278          |
| 1.7871        | 0.71  | 112000 | 2.0588          |
| 1.779         | 0.72  | 114000 | 2.0062          |
| 1.7784        | 0.73  | 116000 | 2.0300          |
| 1.7749        | 0.75  | 118000 | 1.9664          |
| 1.7691        | 0.76  | 120000 | 2.0033          |
| 1.7622        | 0.77  | 122000 | 1.9983          |
| 1.7587        | 0.78  | 124000 | 2.0030          |
| 1.755         | 0.8   | 126000 | 1.9955          |
| 1.7531        | 0.81  | 128000 | 1.9764          |
| 1.7439        | 0.82  | 130000 | 1.9942          |
| 1.7406        | 0.83  | 132000 | 2.0221          |
| 1.7385        | 0.85  | 134000 | 1.9835          |
| 1.7332        | 0.86  | 136000 | 1.9967          |
| 1.7332        | 0.87  | 138000 | 2.0247          |
| 1.7309        | 0.88  | 140000 | 1.9817          |
| 1.7248        | 0.9   | 142000 | 2.0063          |
| 1.7209        | 0.91  | 144000 | 1.9583          |
| 1.7154        | 0.92  | 146000 | 1.9779          |
| 1.7153        | 0.94  | 148000 | 1.9478          |
| 1.7094        | 0.95  | 150000 | 1.9706          |
| 1.7061        | 0.96  | 152000 | 1.9605          |
| 1.7017        | 0.97  | 154000 | 1.9447          |
| 1.6965        | 0.99  | 156000 | 1.9419          |
| 1.6929        | 1.0   | 158000 | 1.9589          |
| 1.6628        | 1.01  | 160000 | 1.9383          |
| 1.6535        | 1.02  | 162000 | 1.9487          |
| 1.6495        | 1.04  | 164000 | 1.9400          |
| 1.6516        | 1.05  | 166000 | 1.9353          |
| 1.6513        | 1.06  | 168000 | 1.9253          |
| 1.6518        | 1.07  | 170000 | 1.9132          |
| 1.6491        | 1.09  | 172000 | 1.9076          |
| 1.6453        | 1.1   | 174000 | 1.9192          |
| 1.6426        | 1.11  | 176000 | 1.9191          |
| 1.6353        | 1.13  | 178000 | 1.9367          |
| 1.6352        | 1.14  | 180000 | 1.9218          |
| 1.6304        | 1.15  | 182000 | 1.9305          |
| 1.6299        | 1.16  | 184000 | 1.9072          |
| 1.6263        | 1.18  | 186000 | 1.9211          |
| 1.6284        | 1.19  | 188000 | 1.9037          |
| 1.6237        | 1.2   | 190000 | 1.8951          |
| 1.6231        | 1.21  | 192000 | 1.8998          |
| 1.6184        | 1.23  | 194000 | 1.8960          |
| 1.6153        | 1.24  | 196000 | 1.8776          |
| 1.6122        | 1.25  | 198000 | 1.8747          |
| 1.6109        | 1.26  | 200000 | 1.8951          |
| 1.6072        | 1.28  | 202000 | 1.8705          |
| 1.6094        | 1.29  | 204000 | 1.8903          |
| 1.6063        | 1.3   | 206000 | 1.8660          |
| 1.599         | 1.31  | 208000 | 1.8696          |
| 1.5931        | 1.33  | 210000 | 1.8598          |
| 1.5943        | 1.34  | 212000 | 1.8760          |
| 1.5906        | 1.35  | 214000 | 1.8833          |
| 1.5858        | 1.37  | 216000 | 1.8645          |
| 1.5873        | 1.38  | 218000 | 1.8620          |
| 1.5842        | 1.39  | 220000 | 1.8632          |
| 1.5808        | 1.4   | 222000 | 1.8782          |
| 1.5756        | 1.42  | 224000 | 1.8627          |
| 1.5728        | 1.43  | 226000 | 1.8649          |
| 1.5709        | 1.44  | 228000 | 1.8735          |
| 1.5704        | 1.45  | 230000 | 1.8630          |
| 1.5659        | 1.47  | 232000 | 1.8598          |
| 1.5637        | 1.48  | 234000 | 1.8519          |
| 1.5628        | 1.49  | 236000 | 1.8569          |
| 1.5559        | 1.5   | 238000 | 1.8401          |
| 1.5532        | 1.52  | 240000 | 1.8528          |
| 1.557         | 1.53  | 242000 | 1.8637          |
| 1.5499        | 1.54  | 244000 | 1.8701          |
| 1.5476        | 1.55  | 246000 | 1.8423          |
| 1.5502        | 1.57  | 248000 | 1.8320          |
| 1.5469        | 1.58  | 250000 | 1.8542          |
| 1.5382        | 1.59  | 252000 | 1.8526          |
| 1.5396        | 1.61  | 254000 | 1.8537          |
| 1.528         | 1.62  | 256000 | 1.8248          |
| 1.532         | 1.63  | 258000 | 1.8322          |
| 1.5269        | 1.64  | 260000 | 1.8381          |
| 1.5269        | 1.66  | 262000 | 1.8389          |
| 1.5269        | 1.67  | 264000 | 1.8445          |
| 1.525         | 1.68  | 266000 | 1.8232          |
| 1.5175        | 1.69  | 268000 | 1.8561          |
| 1.5172        | 1.71  | 270000 | 1.8342          |
| 1.5174        | 1.72  | 272000 | 1.8167          |
| 1.5114        | 1.73  | 274000 | 1.8281          |
| 1.5094        | 1.74  | 276000 | 1.8164          |
| 1.5083        | 1.76  | 278000 | 1.8317          |
| 1.5047        | 1.77  | 280000 | 1.8207          |
| 1.5045        | 1.78  | 282000 | 1.8155          |
| 1.497         | 1.8   | 284000 | 1.8275          |
| 1.4996        | 1.81  | 286000 | 1.8152          |
| 1.497         | 1.82  | 288000 | 1.8137          |
| 1.4967        | 1.83  | 290000 | 1.8109          |
| 1.4936        | 1.85  | 292000 | 1.8037          |
| 1.4867        | 1.86  | 294000 | 1.7955          |
| 1.4859        | 1.87  | 296000 | 1.8181          |
| 1.4869        | 1.88  | 298000 | 1.7999          |
| 1.4811        | 1.9   | 300000 | 1.8062          |
| 1.4831        | 1.91  | 302000 | 1.8042          |
| 1.4791        | 1.92  | 304000 | 1.8020          |
| 1.4797        | 1.93  | 306000 | 1.7972          |
| 1.483         | 1.95  | 308000 | 1.8044          |
| 1.4748        | 1.96  | 310000 | 1.8036          |
| 1.4772        | 1.97  | 312000 | 1.7958          |
| 1.4708        | 1.98  | 314000 | 1.7967          |
| 1.4743        | 2.0   | 316000 | 1.7947          |


### Framework versions

- Transformers 4.19.2
- Pytorch 1.11.0+cu113
- Datasets 2.2.2
- Tokenizers 0.12.1
