pytorch wav2vec2 automatic-speech-recognition dataset:common_voice transformers generated_from_trainer license:apache-2.0 model-index
---
license: apache-2.0
tags:
- generated_from_trainer
datasets:
- common_voice
model-index:
- name: 20220415-210530
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# 20220415-210530

This model is a fine-tuned version of [facebook/wav2vec2-xls-r-2b](https://huggingface.co/facebook/wav2vec2-xls-r-2b) on the common_voice dataset.
It achieves the following results on the evaluation set:
- Loss: 0.6544
- Wer: 0.3881

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 1e-05
- train_batch_size: 1
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 8
- total_train_batch_size: 8
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 400
- num_epochs: 1200
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch   | Step   | Validation Loss | Wer    |
|:-------------:|:-------:|:------:|:---------------:|:------:|
| 6.1495        | 2.27    | 200    | 2.4098          | 1.0    |
| 0.4347        | 4.54    | 400    | 1.4211          | 0.9914 |
| 0.2295        | 6.82    | 600    | 1.0229          | 0.9349 |
| 0.1349        | 9.09    | 800    | 1.0063          | 0.9228 |
| 0.1001        | 11.36   | 1000   | 1.0333          | 0.9197 |
| 0.0847        | 13.63   | 1200   | 0.9021          | 0.8725 |
| 0.0697        | 15.91   | 1400   | 0.9117          | 0.8779 |
| 0.0634        | 18.18   | 1600   | 0.9550          | 0.8725 |
| 0.0607        | 20.45   | 1800   | 0.9063          | 0.8303 |
| 0.0551        | 22.73   | 2000   | 0.8163          | 0.7956 |
| 0.0536        | 25.0    | 2200   | 0.7385          | 0.7235 |
| 0.0511        | 27.27   | 2400   | 0.7917          | 0.7215 |
| 0.0449        | 29.54   | 2600   | 0.7508          | 0.6938 |
| 0.0417        | 31.82   | 2800   | 0.6892          | 0.6775 |
| 0.0415        | 34.09   | 3000   | 0.7029          | 0.6790 |
| 0.0384        | 36.36   | 3200   | 0.6839          | 0.6895 |
| 0.0392        | 38.63   | 3400   | 0.7067          | 0.6872 |
| 0.0358        | 40.91   | 3600   | 0.7310          | 0.6763 |
| 0.0337        | 43.18   | 3800   | 0.7139          | 0.6548 |
| 0.0362        | 45.45   | 4000   | 0.6975          | 0.6427 |
| 0.0311        | 47.73   | 4200   | 0.7054          | 0.6412 |
| 0.0327        | 50.0    | 4400   | 0.6530          | 0.6151 |
| 0.0286        | 52.27   | 4600   | 0.6565          | 0.6076 |
| 0.0304        | 54.54   | 4800   | 0.6931          | 0.6283 |
| 0.0285        | 56.82   | 5000   | 0.6966          | 0.6108 |
| 0.0279        | 59.09   | 5200   | 0.6473          | 0.5854 |
| 0.0276        | 61.36   | 5400   | 0.6497          | 0.5920 |
| 0.0238        | 63.63   | 5600   | 0.6283          | 0.5846 |
| 0.0237        | 65.91   | 5800   | 0.6871          | 0.5885 |
| 0.0221        | 68.18   | 6000   | 0.6518          | 0.5593 |
| 0.0221        | 70.45   | 6200   | 0.6676          | 0.5601 |
| 0.0215        | 72.73   | 6400   | 0.6299          | 0.5550 |
| 0.022         | 75.0    | 6600   | 0.6719          | 0.5636 |
| 0.0198        | 77.27   | 6800   | 0.6082          | 0.5569 |
| 0.0222        | 79.54   | 7000   | 0.6156          | 0.5589 |
| 0.0172        | 81.82   | 7200   | 0.6414          | 0.5636 |
| 0.0188        | 84.09   | 7400   | 0.5874          | 0.5347 |
| 0.0202        | 86.36   | 7600   | 0.6320          | 0.5421 |
| 0.0165        | 88.63   | 7800   | 0.6345          | 0.5304 |
| 0.0164        | 90.91   | 8000   | 0.6243          | 0.5289 |
| 0.0167        | 93.18   | 8200   | 0.6237          | 0.5285 |
| 0.015         | 95.45   | 8400   | 0.5937          | 0.5203 |
| 0.0169        | 97.73   | 8600   | 0.6171          | 0.5343 |
| 0.0147        | 100.0   | 8800   | 0.6857          | 0.5476 |
| 0.0164        | 102.27  | 9000   | 0.6099          | 0.5160 |
| 0.0152        | 104.54  | 9200   | 0.6319          | 0.5285 |
| 0.0149        | 106.82  | 9400   | 0.6133          | 0.5296 |
| 0.0155        | 109.09  | 9600   | 0.6237          | 0.5285 |
| 0.0149        | 111.36  | 9800   | 0.6127          | 0.5012 |
| 0.0142        | 113.63  | 10000  | 0.6119          | 0.4836 |
| 0.013         | 115.91  | 10200  | 0.5974          | 0.4746 |
| 0.012         | 118.18  | 10400  | 0.6296          | 0.5016 |
| 0.0137        | 120.45  | 10600  | 0.5990          | 0.5023 |
| 0.0146        | 122.73  | 10800  | 0.5784          | 0.4875 |
| 0.0117        | 125.0   | 11000  | 0.5436          | 0.4766 |
| 0.0133        | 127.27  | 11200  | 0.5890          | 0.5020 |
| 0.0133        | 129.54  | 11400  | 0.6028          | 0.4895 |
| 0.0119        | 131.82  | 11600  | 0.5483          | 0.4840 |
| 0.0133        | 134.09  | 11800  | 0.5638          | 0.4934 |
| 0.0108        | 136.36  | 12000  | 0.5750          | 0.4758 |
| 0.0098        | 138.63  | 12200  | 0.5978          | 0.4891 |
| 0.012         | 140.91  | 12400  | 0.5524          | 0.4805 |
| 0.01          | 143.18  | 12600  | 0.5731          | 0.4895 |
| 0.0125        | 145.45  | 12800  | 0.5583          | 0.4579 |
| 0.0102        | 147.73  | 13000  | 0.5806          | 0.5035 |
| 0.01          | 150.0   | 13200  | 0.5721          | 0.4711 |
| 0.0113        | 152.27  | 13400  | 0.5351          | 0.4602 |
| 0.011         | 154.54  | 13600  | 0.5472          | 0.4551 |
| 0.0078        | 156.82  | 13800  | 0.6011          | 0.4610 |
| 0.0105        | 159.09  | 14000  | 0.5702          | 0.4672 |
| 0.0081        | 161.36  | 14200  | 0.5643          | 0.4454 |
| 0.0088        | 163.63  | 14400  | 0.5084          | 0.4536 |
| 0.0094        | 165.91  | 14600  | 0.5320          | 0.4680 |
| 0.0083        | 168.18  | 14800  | 0.5175          | 0.4423 |
| 0.0095        | 170.45  | 15000  | 0.5213          | 0.4583 |
| 0.0097        | 172.73  | 15200  | 0.5242          | 0.4590 |
| 0.0092        | 175.0   | 15400  | 0.5680          | 0.4587 |
| 0.0081        | 177.27  | 15600  | 0.5668          | 0.4579 |
| 0.0075        | 179.54  | 15800  | 0.5602          | 0.4489 |
| 0.0094        | 181.82  | 16000  | 0.5540          | 0.4485 |
| 0.0083        | 184.09  | 16200  | 0.5367          | 0.4278 |
| 0.0084        | 186.36  | 16400  | 0.5376          | 0.4583 |
| 0.0093        | 188.63  | 16600  | 0.5599          | 0.4310 |
| 0.0085        | 190.91  | 16800  | 0.5356          | 0.4317 |
| 0.0066        | 193.18  | 17000  | 0.5517          | 0.4419 |
| 0.0074        | 195.45  | 17200  | 0.5401          | 0.4329 |
| 0.0094        | 197.73  | 17400  | 0.5067          | 0.4415 |
| 0.0078        | 200.0   | 17600  | 0.5410          | 0.4466 |
| 0.0085        | 202.27  | 17800  | 0.5157          | 0.4321 |
| 0.0081        | 204.54  | 18000  | 0.5390          | 0.4255 |
| 0.0068        | 206.82  | 18200  | 0.5566          | 0.4415 |
| 0.0069        | 209.09  | 18400  | 0.5693          | 0.4341 |
| 0.0089        | 211.36  | 18600  | 0.5588          | 0.4438 |
| 0.0086        | 213.63  | 18800  | 0.5656          | 0.4470 |
| 0.008         | 215.91  | 19000  | 0.5712          | 0.4438 |
| 0.0083        | 218.18  | 19200  | 0.5627          | 0.4423 |
| 0.0078        | 220.45  | 19400  | 0.5905          | 0.4298 |
| 0.0059        | 222.73  | 19600  | 0.5746          | 0.4228 |
| 0.0072        | 225.0   | 19800  | 0.5362          | 0.4275 |
| 0.006         | 227.27  | 20000  | 0.5909          | 0.4220 |
| 0.0074        | 229.54  | 20200  | 0.5863          | 0.4224 |
| 0.0079        | 231.82  | 20400  | 0.5366          | 0.4306 |
| 0.0066        | 234.09  | 20600  | 0.5128          | 0.4302 |
| 0.0068        | 236.36  | 20800  | 0.5436          | 0.4228 |
| 0.0073        | 238.63  | 21000  | 0.5731          | 0.4325 |
| 0.0081        | 240.91  | 21200  | 0.5189          | 0.4177 |
| 0.0061        | 243.18  | 21400  | 0.5593          | 0.4236 |
| 0.0061        | 245.45  | 21600  | 0.5553          | 0.4267 |
| 0.0044        | 247.73  | 21800  | 0.5763          | 0.4286 |
| 0.0064        | 250.0   | 22000  | 0.5360          | 0.4321 |
| 0.006         | 252.27  | 22200  | 0.5577          | 0.4372 |
| 0.0052        | 254.54  | 22400  | 0.5387          | 0.4122 |
| 0.0054        | 256.82  | 22600  | 0.5117          | 0.4239 |
| 0.0057        | 259.09  | 22800  | 0.5498          | 0.4232 |
| 0.0069        | 261.36  | 23000  | 0.5263          | 0.4353 |
| 0.005         | 263.63  | 23200  | 0.5147          | 0.4177 |
| 0.0058        | 265.91  | 23400  | 0.5273          | 0.4173 |
| 0.006         | 268.18  | 23600  | 0.5879          | 0.4380 |
| 0.0059        | 270.45  | 23800  | 0.5377          | 0.4349 |
| 0.0055        | 272.73  | 24000  | 0.6061          | 0.4364 |
| 0.0058        | 275.0   | 24200  | 0.5977          | 0.4353 |
| 0.0051        | 277.27  | 24400  | 0.5847          | 0.4208 |
| 0.0046        | 279.54  | 24600  | 0.5728          | 0.4333 |
| 0.006         | 281.82  | 24800  | 0.5392          | 0.4204 |
| 0.0074        | 284.09  | 25000  | 0.5618          | 0.4232 |
| 0.0058        | 286.36  | 25200  | 0.5449          | 0.4197 |
| 0.0057        | 288.63  | 25400  | 0.5635          | 0.4169 |
| 0.0054        | 290.91  | 25600  | 0.5313          | 0.4173 |
| 0.0044        | 293.18  | 25800  | 0.5544          | 0.4306 |
| 0.0039        | 295.45  | 26000  | 0.5392          | 0.4247 |
| 0.0054        | 297.73  | 26200  | 0.5395          | 0.4271 |
| 0.0044        | 300.0   | 26400  | 0.5489          | 0.4228 |
| 0.0042        | 302.27  | 26600  | 0.5414          | 0.4173 |
| 0.0051        | 304.54  | 26800  | 0.5198          | 0.4193 |
| 0.005         | 306.82  | 27000  | 0.5297          | 0.4146 |
| 0.0051        | 309.09  | 27200  | 0.5414          | 0.4212 |
| 0.0057        | 311.36  | 27400  | 0.5204          | 0.4228 |
| 0.0049        | 313.63  | 27600  | 0.5806          | 0.4239 |
| 0.0036        | 315.91  | 27800  | 0.5771          | 0.4173 |
| 0.0045        | 318.18  | 28000  | 0.5517          | 0.4239 |
| 0.0051        | 320.45  | 28200  | 0.5498          | 0.4173 |
| 0.0043        | 322.73  | 28400  | 0.5791          | 0.4181 |
| 0.0044        | 325.0   | 28600  | 0.6030          | 0.4200 |
| 0.0067        | 327.27  | 28800  | 0.5799          | 0.4208 |
| 0.0041        | 329.54  | 29000  | 0.5871          | 0.4134 |
| 0.0048        | 331.82  | 29200  | 0.5471          | 0.4158 |
| 0.0031        | 334.09  | 29400  | 0.5977          | 0.4220 |
| 0.0042        | 336.36  | 29600  | 0.5813          | 0.4181 |
| 0.0045        | 338.63  | 29800  | 0.6167          | 0.4306 |
| 0.0044        | 340.91  | 30000  | 0.5661          | 0.4173 |
| 0.0029        | 343.18  | 30200  | 0.5680          | 0.4158 |
| 0.0037        | 345.45  | 30400  | 0.5747          | 0.4204 |
| 0.005         | 347.73  | 30600  | 0.5883          | 0.4349 |
| 0.0037        | 350.0   | 30800  | 0.6187          | 0.4189 |
| 0.0044        | 352.27  | 31000  | 0.5834          | 0.4431 |
| 0.0047        | 354.54  | 31200  | 0.5567          | 0.4247 |
| 0.0039        | 356.82  | 31400  | 0.5900          | 0.4314 |
| 0.0044        | 359.09  | 31600  | 0.5879          | 0.4216 |
| 0.0042        | 361.36  | 31800  | 0.5639          | 0.4220 |
| 0.0046        | 363.63  | 32000  | 0.5292          | 0.4185 |
| 0.0043        | 365.91  | 32200  | 0.5640          | 0.4353 |
| 0.0033        | 368.18  | 32400  | 0.5468          | 0.4208 |
| 0.002         | 370.45  | 32600  | 0.5836          | 0.4220 |
| 0.0043        | 372.73  | 32800  | 0.5692          | 0.4142 |
| 0.0038        | 375.0   | 33000  | 0.5739          | 0.4177 |
| 0.0039        | 377.27  | 33200  | 0.5824          | 0.4103 |
| 0.0028        | 379.54  | 33400  | 0.6069          | 0.4111 |
| 0.0038        | 381.82  | 33600  | 0.5868          | 0.4185 |
| 0.0041        | 384.09  | 33800  | 0.5169          | 0.4126 |
| 0.0037        | 386.36  | 34000  | 0.5395          | 0.4275 |
| 0.0063        | 388.63  | 34200  | 0.5293          | 0.4294 |
| 0.0042        | 390.91  | 34400  | 0.5472          | 0.4165 |
| 0.0039        | 393.18  | 34600  | 0.5391          | 0.4091 |
| 0.0036        | 395.45  | 34800  | 0.5360          | 0.4239 |
| 0.0036        | 397.73  | 35000  | 0.5511          | 0.4177 |
| 0.0019        | 400.0   | 35200  | 0.5775          | 0.4115 |
| 0.0038        | 402.27  | 35400  | 0.5376          | 0.4087 |
| 0.0035        | 404.54  | 35600  | 0.5755          | 0.4130 |
| 0.0042        | 406.82  | 35800  | 0.5443          | 0.4087 |
| 0.0036        | 409.09  | 36000  | 0.6091          | 0.4200 |
| 0.004         | 411.36  | 36200  | 0.5817          | 0.4247 |
| 0.0039        | 413.63  | 36400  | 0.5779          | 0.4255 |
| 0.003         | 415.91  | 36600  | 0.5804          | 0.4224 |
| 0.0031        | 418.18  | 36800  | 0.5467          | 0.4138 |
| 0.0044        | 420.45  | 37000  | 0.5628          | 0.4212 |
| 0.0036        | 422.73  | 37200  | 0.5613          | 0.4267 |
| 0.0035        | 425.0   | 37400  | 0.5537          | 0.4224 |
| 0.0028        | 427.27  | 37600  | 0.6016          | 0.4161 |
| 0.004         | 429.54  | 37800  | 0.5711          | 0.4216 |
| 0.0041        | 431.82  | 38000  | 0.5510          | 0.4165 |
| 0.0035        | 434.09  | 38200  | 0.5487          | 0.4181 |
| 0.0034        | 436.36  | 38400  | 0.5392          | 0.4056 |
| 0.003         | 438.63  | 38600  | 0.5255          | 0.4083 |
| 0.0035        | 440.91  | 38800  | 0.5511          | 0.4138 |
| 0.0031        | 443.18  | 39000  | 0.5464          | 0.4146 |
| 0.0032        | 445.45  | 39200  | 0.5514          | 0.4134 |
| 0.0017        | 447.73  | 39400  | 0.5664          | 0.4064 |
| 0.0024        | 450.0   | 39600  | 0.5966          | 0.4220 |
| 0.0021        | 452.27  | 39800  | 0.5780          | 0.4122 |
| 0.0035        | 454.54  | 40000  | 0.5612          | 0.4341 |
| 0.002         | 456.82  | 40200  | 0.5954          | 0.4247 |
| 0.0018        | 459.09  | 40400  | 0.6006          | 0.4251 |
| 0.0026        | 461.36  | 40600  | 0.6119          | 0.4232 |
| 0.0023        | 463.63  | 40800  | 0.6051          | 0.4306 |
| 0.003         | 465.91  | 41000  | 0.5872          | 0.4267 |
| 0.0036        | 468.18  | 41200  | 0.5602          | 0.4095 |
| 0.0029        | 470.45  | 41400  | 0.5877          | 0.4189 |
| 0.0034        | 472.73  | 41600  | 0.5918          | 0.4337 |
| 0.0025        | 475.0   | 41800  | 0.6101          | 0.4337 |
| 0.0023        | 477.27  | 42000  | 0.5936          | 0.4239 |
| 0.0017        | 479.54  | 42200  | 0.6257          | 0.4275 |
| 0.0029        | 481.82  | 42400  | 0.6265          | 0.4251 |
| 0.0035        | 484.09  | 42600  | 0.6035          | 0.4271 |
| 0.0036        | 486.36  | 42800  | 0.5954          | 0.4243 |
| 0.0028        | 488.63  | 43000  | 0.5810          | 0.4259 |
| 0.0027        | 490.91  | 43200  | 0.6093          | 0.4228 |
| 0.0025        | 493.18  | 43400  | 0.6241          | 0.4302 |
| 0.0019        | 495.45  | 43600  | 0.6143          | 0.4290 |
| 0.0025        | 497.73  | 43800  | 0.5729          | 0.4189 |
| 0.0028        | 500.0   | 44000  | 0.5725          | 0.4165 |
| 0.0023        | 502.27  | 44200  | 0.5888          | 0.4263 |
| 0.0034        | 504.54  | 44400  | 0.5771          | 0.4337 |
| 0.0022        | 506.82  | 44600  | 0.5888          | 0.4216 |
| 0.0028        | 509.09  | 44800  | 0.5598          | 0.4181 |
| 0.0024        | 511.36  | 45000  | 0.6114          | 0.4392 |
| 0.0037        | 513.63  | 45200  | 0.5855          | 0.4236 |
| 0.0018        | 515.91  | 45400  | 0.5885          | 0.4232 |
| 0.0025        | 518.18  | 45600  | 0.5845          | 0.4255 |
| 0.0029        | 520.45  | 45800  | 0.5862          | 0.4380 |
| 0.0034        | 522.73  | 46000  | 0.5807          | 0.4329 |
| 0.0025        | 525.0   | 46200  | 0.5959          | 0.4189 |
| 0.0025        | 527.27  | 46400  | 0.5939          | 0.4216 |
| 0.0022        | 529.54  | 46600  | 0.5964          | 0.4232 |
| 0.003         | 531.82  | 46800  | 0.5664          | 0.4173 |
| 0.0021        | 534.09  | 47000  | 0.5670          | 0.4138 |
| 0.0025        | 536.36  | 47200  | 0.5611          | 0.4247 |
| 0.0024        | 538.63  | 47400  | 0.5691          | 0.4321 |
| 0.0019        | 540.91  | 47600  | 0.5992          | 0.4224 |
| 0.0037        | 543.18  | 47800  | 0.5790          | 0.4181 |
| 0.0025        | 545.45  | 48000  | 0.5650          | 0.4294 |
| 0.0025        | 547.73  | 48200  | 0.5732          | 0.4189 |
| 0.0025        | 550.0   | 48400  | 0.5566          | 0.4220 |
| 0.0023        | 552.27  | 48600  | 0.5646          | 0.4236 |
| 0.0027        | 554.54  | 48800  | 0.5437          | 0.4263 |
| 0.0026        | 556.82  | 49000  | 0.5993          | 0.4239 |
| 0.0017        | 559.09  | 49200  | 0.6158          | 0.4212 |
| 0.002         | 561.36  | 49400  | 0.6104          | 0.4064 |
| 0.0028        | 563.63  | 49600  | 0.5689          | 0.4021 |
| 0.0025        | 565.91  | 49800  | 0.5760          | 0.4029 |
| 0.0024        | 568.18  | 50000  | 0.5700          | 0.4037 |
| 0.0024        | 570.45  | 50200  | 0.5509          | 0.3935 |
| 0.0018        | 572.73  | 50400  | 0.5562          | 0.4048 |
| 0.0018        | 575.0   | 50600  | 0.5786          | 0.3955 |
| 0.0023        | 577.27  | 50800  | 0.5855          | 0.3959 |
| 0.0017        | 579.54  | 51000  | 0.5988          | 0.3939 |
| 0.0021        | 581.82  | 51200  | 0.6132          | 0.4064 |
| 0.0017        | 584.09  | 51400  | 0.6202          | 0.4099 |
| 0.0019        | 586.36  | 51600  | 0.6118          | 0.4048 |
| 0.0023        | 588.63  | 51800  | 0.6114          | 0.4158 |
| 0.0019        | 590.91  | 52000  | 0.5808          | 0.4126 |
| 0.0025        | 593.18  | 52200  | 0.5906          | 0.4037 |
| 0.0016        | 595.45  | 52400  | 0.5965          | 0.4056 |
| 0.0021        | 597.73  | 52600  | 0.6126          | 0.4099 |
| 0.0019        | 600.0   | 52800  | 0.5913          | 0.4060 |
| 0.0014        | 602.27  | 53000  | 0.6450          | 0.4076 |
| 0.0021        | 604.54  | 53200  | 0.6500          | 0.4189 |
| 0.002         | 606.82  | 53400  | 0.6026          | 0.4111 |
| 0.0022        | 609.09  | 53600  | 0.6318          | 0.4099 |
| 0.003         | 611.36  | 53800  | 0.6038          | 0.4111 |
| 0.0022        | 613.63  | 54000  | 0.6086          | 0.4083 |
| 0.0013        | 615.91  | 54200  | 0.6320          | 0.4025 |
| 0.0016        | 618.18  | 54400  | 0.6159          | 0.3974 |
| 0.0018        | 620.45  | 54600  | 0.6266          | 0.3998 |
| 0.002         | 622.73  | 54800  | 0.5920          | 0.3994 |
| 0.001         | 625.0   | 55000  | 0.6196          | 0.3935 |
| 0.0018        | 627.27  | 55200  | 0.6468          | 0.4009 |
| 0.002         | 629.54  | 55400  | 0.6505          | 0.4052 |
| 0.002         | 631.82  | 55600  | 0.6362          | 0.4072 |
| 0.0018        | 634.09  | 55800  | 0.6430          | 0.3963 |
| 0.0017        | 636.36  | 56000  | 0.6434          | 0.3966 |
| 0.0014        | 638.63  | 56200  | 0.6473          | 0.4080 |
| 0.0021        | 640.91  | 56400  | 0.6272          | 0.4115 |
| 0.0026        | 643.18  | 56600  | 0.6343          | 0.4099 |
| 0.0023        | 645.45  | 56800  | 0.6223          | 0.4025 |
| 0.0016        | 647.73  | 57000  | 0.5879          | 0.4025 |
| 0.001         | 650.0   | 57200  | 0.6274          | 0.4005 |
| 0.0019        | 652.27  | 57400  | 0.6517          | 0.4044 |
| 0.0011        | 654.54  | 57600  | 0.6571          | 0.4080 |
| 0.002         | 656.82  | 57800  | 0.6377          | 0.4087 |
| 0.0024        | 659.09  | 58000  | 0.6013          | 0.4146 |
| 0.0021        | 661.36  | 58200  | 0.5985          | 0.4185 |
| 0.0018        | 663.63  | 58400  | 0.6148          | 0.4150 |
| 0.0015        | 665.91  | 58600  | 0.6318          | 0.4013 |
| 0.0016        | 668.18  | 58800  | 0.6109          | 0.4025 |
| 0.002         | 670.45  | 59000  | 0.5823          | 0.4029 |
| 0.0013        | 672.73  | 59200  | 0.5800          | 0.4146 |
| 0.0018        | 675.0   | 59400  | 0.5794          | 0.4080 |
| 0.0012        | 677.27  | 59600  | 0.5997          | 0.4037 |
| 0.0016        | 679.54  | 59800  | 0.6111          | 0.4005 |
| 0.0019        | 681.82  | 60000  | 0.6112          | 0.4099 |
| 0.0022        | 684.09  | 60200  | 0.6030          | 0.4068 |
| 0.0013        | 686.36  | 60400  | 0.6247          | 0.4115 |
| 0.0017        | 688.63  | 60600  | 0.5981          | 0.4111 |
| 0.0016        | 690.91  | 60800  | 0.5773          | 0.4122 |
| 0.0016        | 693.18  | 61000  | 0.6019          | 0.4068 |
| 0.0014        | 695.45  | 61200  | 0.5931          | 0.4021 |
| 0.0015        | 697.73  | 61400  | 0.6391          | 0.4083 |
| 0.0015        | 700.0   | 61600  | 0.6148          | 0.4021 |
| 0.0013        | 702.27  | 61800  | 0.6143          | 0.4138 |
| 0.0009        | 704.54  | 62000  | 0.6203          | 0.4115 |
| 0.0015        | 706.82  | 62200  | 0.6452          | 0.4115 |
| 0.0011        | 709.09  | 62400  | 0.6323          | 0.4107 |
| 0.0025        | 711.36  | 62600  | 0.6248          | 0.4243 |
| 0.001         | 713.63  | 62800  | 0.6225          | 0.4189 |
| 0.0013        | 715.91  | 63000  | 0.6328          | 0.4161 |
| 0.0011        | 718.18  | 63200  | 0.6299          | 0.4130 |
| 0.0016        | 720.45  | 63400  | 0.6110          | 0.4072 |
| 0.0012        | 722.73  | 63600  | 0.6095          | 0.4064 |
| 0.0017        | 725.0   | 63800  | 0.6205          | 0.4033 |
| 0.0009        | 727.27  | 64000  | 0.6330          | 0.4099 |
| 0.0011        | 729.54  | 64200  | 0.6184          | 0.3974 |
| 0.0016        | 731.82  | 64400  | 0.6147          | 0.4052 |
| 0.0014        | 734.09  | 64600  | 0.6271          | 0.4068 |
| 0.0013        | 736.36  | 64800  | 0.6157          | 0.4091 |
| 0.0017        | 738.63  | 65000  | 0.6157          | 0.4072 |
| 0.0022        | 740.91  | 65200  | 0.5888          | 0.4177 |
| 0.0017        | 743.18  | 65400  | 0.6002          | 0.4134 |
| 0.0017        | 745.45  | 65600  | 0.5989          | 0.4161 |
| 0.0016        | 747.73  | 65800  | 0.6069          | 0.4185 |
| 0.0019        | 750.0   | 66000  | 0.5962          | 0.4212 |
| 0.0011        | 752.27  | 66200  | 0.6044          | 0.4161 |
| 0.0014        | 754.54  | 66400  | 0.5978          | 0.4197 |
| 0.0008        | 756.82  | 66600  | 0.6291          | 0.4146 |
| 0.0009        | 759.09  | 66800  | 0.6203          | 0.4181 |
| 0.0009        | 761.36  | 67000  | 0.6124          | 0.4138 |
| 0.0013        | 763.63  | 67200  | 0.6191          | 0.4138 |
| 0.0017        | 765.91  | 67400  | 0.6061          | 0.4087 |
| 0.001         | 768.18  | 67600  | 0.6233          | 0.4111 |
| 0.0014        | 770.45  | 67800  | 0.6189          | 0.4080 |
| 0.0013        | 772.73  | 68000  | 0.6493          | 0.4056 |
| 0.0013        | 775.0   | 68200  | 0.6454          | 0.4037 |
| 0.0013        | 777.27  | 68400  | 0.6373          | 0.4095 |
| 0.0011        | 779.54  | 68600  | 0.6563          | 0.4041 |
| 0.0013        | 781.82  | 68800  | 0.6622          | 0.4122 |
| 0.0012        | 784.09  | 69000  | 0.6858          | 0.4220 |
| 0.0019        | 786.36  | 69200  | 0.6658          | 0.4126 |
| 0.001         | 788.63  | 69400  | 0.6650          | 0.4068 |
| 0.0007        | 790.91  | 69600  | 0.6777          | 0.4107 |
| 0.0011        | 793.18  | 69800  | 0.6772          | 0.4158 |
| 0.001         | 795.45  | 70000  | 0.6820          | 0.4173 |
| 0.0007        | 797.73  | 70200  | 0.6870          | 0.4138 |
| 0.0011        | 800.0   | 70400  | 0.6732          | 0.4115 |
| 0.0011        | 802.27  | 70600  | 0.6755          | 0.4154 |
| 0.0009        | 804.54  | 70800  | 0.6707          | 0.4224 |
| 0.0014        | 806.82  | 71000  | 0.6733          | 0.4134 |
| 0.0009        | 809.09  | 71200  | 0.6690          | 0.4142 |
| 0.0011        | 811.36  | 71400  | 0.6875          | 0.4169 |
| 0.0019        | 813.63  | 71600  | 0.6471          | 0.4138 |
| 0.0006        | 815.91  | 71800  | 0.6599          | 0.4099 |
| 0.0014        | 818.18  | 72000  | 0.6543          | 0.4052 |
| 0.0011        | 820.45  | 72200  | 0.6699          | 0.4052 |
| 0.0014        | 822.73  | 72400  | 0.6626          | 0.4080 |
| 0.0014        | 825.0   | 72600  | 0.6601          | 0.4142 |
| 0.0007        | 827.27  | 72800  | 0.6686          | 0.4115 |
| 0.0007        | 829.54  | 73000  | 0.6657          | 0.4134 |
| 0.0009        | 831.82  | 73200  | 0.6810          | 0.4056 |
| 0.0013        | 834.09  | 73400  | 0.6734          | 0.4060 |
| 0.0005        | 836.36  | 73600  | 0.6815          | 0.4033 |
| 0.0026        | 838.63  | 73800  | 0.6607          | 0.4056 |
| 0.001         | 840.91  | 74000  | 0.6700          | 0.4041 |
| 0.0008        | 843.18  | 74200  | 0.6871          | 0.4041 |
| 0.0006        | 845.45  | 74400  | 0.6910          | 0.4099 |
| 0.0009        | 847.73  | 74600  | 0.7027          | 0.4064 |
| 0.0009        | 850.0   | 74800  | 0.7108          | 0.4017 |
| 0.0005        | 852.27  | 75000  | 0.7122          | 0.3986 |
| 0.001         | 854.54  | 75200  | 0.7051          | 0.3982 |
| 0.0007        | 856.82  | 75400  | 0.7266          | 0.3978 |
| 0.0015        | 859.09  | 75600  | 0.7051          | 0.4017 |
| 0.0007        | 861.36  | 75800  | 0.7038          | 0.3970 |
| 0.001         | 863.63  | 76000  | 0.6847          | 0.4037 |
| 0.0013        | 865.91  | 76200  | 0.6823          | 0.4033 |
| 0.001         | 868.18  | 76400  | 0.6926          | 0.4060 |
| 0.0018        | 870.45  | 76600  | 0.7035          | 0.4025 |
| 0.0007        | 872.73  | 76800  | 0.6993          | 0.4048 |
| 0.0006        | 875.0   | 77000  | 0.7083          | 0.4048 |
| 0.001         | 877.27  | 77200  | 0.7217          | 0.4083 |
| 0.0014        | 879.54  | 77400  | 0.7013          | 0.4076 |
| 0.0009        | 881.82  | 77600  | 0.6874          | 0.4083 |
| 0.0012        | 884.09  | 77800  | 0.6966          | 0.4103 |
| 0.0008        | 886.36  | 78000  | 0.6989          | 0.3982 |
| 0.001         | 888.63  | 78200  | 0.7000          | 0.4115 |
| 0.0011        | 890.91  | 78400  | 0.7105          | 0.4107 |
| 0.0008        | 893.18  | 78600  | 0.7103          | 0.4068 |
| 0.0022        | 895.45  | 78800  | 0.6641          | 0.4033 |
| 0.0006        | 897.73  | 79000  | 0.6635          | 0.4048 |
| 0.0009        | 900.0   | 79200  | 0.6535          | 0.4072 |
| 0.0009        | 902.27  | 79400  | 0.6598          | 0.4048 |
| 0.0007        | 904.54  | 79600  | 0.6684          | 0.4017 |
| 0.0008        | 906.82  | 79800  | 0.6752          | 0.4009 |
| 0.0008        | 909.09  | 80000  | 0.6820          | 0.4037 |
| 0.0009        | 911.36  | 80200  | 0.6672          | 0.3986 |
| 0.0007        | 913.63  | 80400  | 0.6692          | 0.4025 |
| 0.001         | 915.91  | 80600  | 0.6676          | 0.4056 |
| 0.0012        | 918.18  | 80800  | 0.6484          | 0.4002 |
| 0.0008        | 920.45  | 81000  | 0.6541          | 0.4002 |
| 0.0005        | 922.73  | 81200  | 0.6626          | 0.3990 |
| 0.0013        | 925.0   | 81400  | 0.6688          | 0.3994 |
| 0.0015        | 927.27  | 81600  | 0.6472          | 0.4048 |
| 0.0011        | 929.54  | 81800  | 0.6432          | 0.4041 |
| 0.0012        | 931.82  | 82000  | 0.6374          | 0.3939 |
| 0.0005        | 934.09  | 82200  | 0.6519          | 0.4005 |
| 0.001         | 936.36  | 82400  | 0.6281          | 0.3998 |
| 0.0007        | 938.63  | 82600  | 0.6621          | 0.4048 |
| 0.0005        | 940.91  | 82800  | 0.6670          | 0.3990 |
| 0.0009        | 943.18  | 83000  | 0.6707          | 0.3982 |
| 0.0006        | 945.45  | 83200  | 0.6592          | 0.3924 |
| 0.0006        | 947.73  | 83400  | 0.6772          | 0.4002 |
| 0.0017        | 950.0   | 83600  | 0.6786          | 0.4068 |
| 0.0004        | 952.27  | 83800  | 0.6849          | 0.4052 |
| 0.0002        | 954.54  | 84000  | 0.6914          | 0.4044 |
| 0.0009        | 956.82  | 84200  | 0.6806          | 0.4002 |
| 0.0006        | 959.09  | 84400  | 0.6621          | 0.4013 |
| 0.0004        | 961.36  | 84600  | 0.6712          | 0.4029 |
| 0.0007        | 963.63  | 84800  | 0.6775          | 0.4052 |
| 0.0004        | 965.91  | 85000  | 0.6769          | 0.4080 |
| 0.001         | 968.18  | 85200  | 0.6470          | 0.4029 |
| 0.0009        | 970.45  | 85400  | 0.6505          | 0.4002 |
| 0.0011        | 972.73  | 85600  | 0.6543          | 0.4041 |
| 0.0003        | 975.0   | 85800  | 0.6568          | 0.4009 |
| 0.0004        | 977.27  | 86000  | 0.6627          | 0.3990 |
| 0.0014        | 979.54  | 86200  | 0.6564          | 0.4021 |
| 0.0012        | 981.82  | 86400  | 0.6535          | 0.3982 |
| 0.0007        | 984.09  | 86600  | 0.6443          | 0.4009 |
| 0.0008        | 986.36  | 86800  | 0.6466          | 0.4005 |
| 0.0004        | 988.63  | 87000  | 0.6538          | 0.4017 |
| 0.0008        | 990.91  | 87200  | 0.6485          | 0.3998 |
| 0.0004        | 993.18  | 87400  | 0.6504          | 0.3951 |
| 0.0008        | 995.45  | 87600  | 0.6410          | 0.3970 |
| 0.0004        | 997.73  | 87800  | 0.6420          | 0.3986 |
| 0.0005        | 1000.0  | 88000  | 0.6507          | 0.3998 |
| 0.0005        | 1002.27 | 88200  | 0.6540          | 0.3998 |
| 0.0006        | 1004.54 | 88400  | 0.6531          | 0.3978 |
| 0.0015        | 1006.82 | 88600  | 0.6411          | 0.3986 |
| 0.0007        | 1009.09 | 88800  | 0.6411          | 0.3990 |
| 0.0003        | 1011.36 | 89000  | 0.6432          | 0.3998 |
| 0.0004        | 1013.63 | 89200  | 0.6546          | 0.4021 |
| 0.0004        | 1015.91 | 89400  | 0.6542          | 0.4002 |
| 0.0006        | 1018.18 | 89600  | 0.6622          | 0.4009 |
| 0.0008        | 1020.45 | 89800  | 0.6674          | 0.3963 |
| 0.0008        | 1022.73 | 90000  | 0.6563          | 0.3935 |
| 0.0003        | 1025.0  | 90200  | 0.6638          | 0.3955 |
| 0.0004        | 1027.27 | 90400  | 0.6667          | 0.3951 |
| 0.001         | 1029.54 | 90600  | 0.6462          | 0.3943 |
| 0.0007        | 1031.82 | 90800  | 0.6462          | 0.3920 |
| 0.0006        | 1034.09 | 91000  | 0.6477          | 0.3947 |
| 0.0005        | 1036.36 | 91200  | 0.6500          | 0.3955 |
| 0.0006        | 1038.63 | 91400  | 0.6461          | 0.3955 |
| 0.0007        | 1040.91 | 91600  | 0.6526          | 0.4002 |
| 0.0004        | 1043.18 | 91800  | 0.6514          | 0.4021 |
| 0.0003        | 1045.45 | 92000  | 0.6610          | 0.4025 |
| 0.0007        | 1047.73 | 92200  | 0.6583          | 0.3966 |
| 0.0004        | 1050.0  | 92400  | 0.6413          | 0.3955 |
| 0.0009        | 1052.27 | 92600  | 0.6411          | 0.3951 |
| 0.0008        | 1054.54 | 92800  | 0.6374          | 0.3978 |
| 0.0003        | 1056.82 | 93000  | 0.6359          | 0.3955 |
| 0.0006        | 1059.09 | 93200  | 0.6400          | 0.3955 |
| 0.0007        | 1061.36 | 93400  | 0.6363          | 0.3974 |
| 0.0002        | 1063.63 | 93600  | 0.6413          | 0.3959 |
| 0.0006        | 1065.91 | 93800  | 0.6428          | 0.3927 |
| 0.0007        | 1068.18 | 94000  | 0.6388          | 0.3912 |
| 0.0007        | 1070.45 | 94200  | 0.6371          | 0.3920 |
| 0.0005        | 1072.73 | 94400  | 0.6449          | 0.3904 |
| 0.0015        | 1075.0  | 94600  | 0.6415          | 0.3916 |
| 0.0005        | 1077.27 | 94800  | 0.6355          | 0.3920 |
| 0.0005        | 1079.54 | 95000  | 0.6362          | 0.3920 |
| 0.0004        | 1081.82 | 95200  | 0.6303          | 0.3931 |
| 0.0011        | 1084.09 | 95400  | 0.6255          | 0.3955 |
| 0.0003        | 1086.36 | 95600  | 0.6314          | 0.3959 |
| 0.0005        | 1088.63 | 95800  | 0.6353          | 0.3943 |
| 0.0007        | 1090.91 | 96000  | 0.6398          | 0.3931 |
| 0.0003        | 1093.18 | 96200  | 0.6472          | 0.3963 |
| 0.0007        | 1095.45 | 96400  | 0.6479          | 0.3947 |
| 0.0005        | 1097.73 | 96600  | 0.6520          | 0.3947 |
| 0.0005        | 1100.0  | 96800  | 0.6569          | 0.3963 |
| 0.0007        | 1102.27 | 97000  | 0.6551          | 0.3982 |
| 0.0004        | 1104.54 | 97200  | 0.6554          | 0.3966 |
| 0.0013        | 1106.82 | 97400  | 0.6404          | 0.3963 |
| 0.0008        | 1109.09 | 97600  | 0.6421          | 0.3963 |
| 0.0007        | 1111.36 | 97800  | 0.6379          | 0.3931 |
| 0.0003        | 1113.63 | 98000  | 0.6403          | 0.3931 |
| 0.0003        | 1115.91 | 98200  | 0.6443          | 0.3916 |
| 0.0004        | 1118.18 | 98400  | 0.6461          | 0.3986 |
| 0.0003        | 1120.45 | 98600  | 0.6440          | 0.3978 |
| 0.0001        | 1122.73 | 98800  | 0.6480          | 0.4002 |
| 0.0006        | 1125.0  | 99000  | 0.6497          | 0.4005 |
| 0.0004        | 1127.27 | 99200  | 0.6533          | 0.4009 |
| 0.0007        | 1129.54 | 99400  | 0.6461          | 0.3978 |
| 0.0006        | 1131.82 | 99600  | 0.6453          | 0.3986 |
| 0.0001        | 1134.09 | 99800  | 0.6478          | 0.4002 |
| 0.0005        | 1136.36 | 100000 | 0.6508          | 0.3978 |
| 0.0004        | 1138.63 | 100200 | 0.6500          | 0.3955 |
| 0.0006        | 1140.91 | 100400 | 0.6521          | 0.3924 |
| 0.0004        | 1143.18 | 100600 | 0.6543          | 0.3931 |
| 0.0004        | 1145.45 | 100800 | 0.6552          | 0.3935 |
| 0.0006        | 1147.73 | 101000 | 0.6550          | 0.3931 |
| 0.0002        | 1150.0  | 101200 | 0.6567          | 0.3924 |
| 0.0002        | 1152.27 | 101400 | 0.6585          | 0.3912 |
| 0.0006        | 1154.54 | 101600 | 0.6588          | 0.3900 |
| 0.0006        | 1156.82 | 101800 | 0.6583          | 0.3892 |
| 0.0007        | 1159.09 | 102000 | 0.6579          | 0.3916 |
| 0.0003        | 1161.36 | 102200 | 0.6588          | 0.3908 |
| 0.0004        | 1163.63 | 102400 | 0.6603          | 0.3912 |
| 0.0004        | 1165.91 | 102600 | 0.6602          | 0.3916 |
| 0.0004        | 1168.18 | 102800 | 0.6596          | 0.3916 |
| 0.0007        | 1170.45 | 103000 | 0.6577          | 0.3924 |
| 0.0003        | 1172.73 | 103200 | 0.6593          | 0.3900 |
| 0.0004        | 1175.0  | 103400 | 0.6577          | 0.3900 |
| 0.0006        | 1177.27 | 103600 | 0.6554          | 0.3900 |
| 0.0004        | 1179.54 | 103800 | 0.6554          | 0.3885 |
| 0.0005        | 1181.82 | 104000 | 0.6545          | 0.3873 |
| 0.0003        | 1184.09 | 104200 | 0.6545          | 0.3885 |
| 0.0002        | 1186.36 | 104400 | 0.6546          | 0.3888 |
| 0.0006        | 1188.63 | 104600 | 0.6547          | 0.3892 |
| 0.0007        | 1190.91 | 104800 | 0.6542          | 0.3885 |
| 0.0002        | 1193.18 | 105000 | 0.6543          | 0.3885 |
| 0.0003        | 1195.45 | 105200 | 0.6544          | 0.3881 |
| 0.0004        | 1197.73 | 105400 | 0.6544          | 0.3881 |
| 0.0009        | 1200.0  | 105600 | 0.6544          | 0.3881 |


### Framework versions

- Transformers 4.18.0.dev0
- Pytorch 1.10.0+cu113
- Datasets 1.18.3
- Tokenizers 0.11.6
