nllb-200-1.3B-ft-eng-to-cym

This model is a fine-tuned version of facebook/nllb-200-1.3B on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.5924
• Bleu: 34.8298
• Gen Len: 43.7448

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 6000
• training_steps: 30000

Training results

Training Loss	Epoch	Step	Validation Loss	Bleu	Gen Len
1.0564	0.0261	2000	0.9273	28.4328	38.8496
0.9194	0.0523	4000	0.8338	31.7143	46.5768
0.8274	0.0784	6000	0.7616	34.7164	53.7005
0.7484	0.1045	8000	0.7126	36.4499	63.7493
0.7263	0.1306	10000	0.6823	33.8987	50.8464
0.6844	0.1568	12000	0.6590	30.4939	56.9974
0.6574	0.1829	14000	0.6480	37.6831	46.6029
0.6522	0.2090	16000	0.6311	33.1157	49.7663
0.6267	0.2351	18000	0.6211	31.5074	50.7949
0.6231	0.2613	20000	0.6120	35.5055	46.3893
0.6101	0.2874	22000	0.6044	34.8104	45.3366
0.5973	0.3135	24000	0.5997	34.3889	43.5215
0.6049	0.3397	26000	0.5961	33.9847	42.8822
0.5949	0.3658	28000	0.5937	35.5082	44.2201
0.5947	0.3919	30000	0.5924	34.8298	43.7448

Framework versions

• Transformers 4.49.0
• Pytorch 2.6.0+cu124
• Datasets 3.3.2
• Tokenizers 0.21.0