distilbert-base-uncased-finetuned-squad

This model is a fine-tuned version of distilbert-base-uncased on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.0629

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 30

Training results

Training Loss	Epoch	Step	Validation Loss
No log	1.0	4	5.6141
No log	2.0	8	4.6457
No log	3.0	12	3.9387
No log	4.0	16	3.1467
No log	5.0	20	2.3827
No log	6.0	24	1.9844
No log	7.0	28	1.7256
No log	8.0	32	1.5568
No log	9.0	36	1.3831
No log	10.0	40	1.2769
No log	11.0	44	1.2855
No log	12.0	48	1.1126
No log	13.0	52	1.1142
No log	14.0	56	1.2023
No log	15.0	60	1.1673
No log	16.0	64	1.1534
No log	17.0	68	1.2009
No log	18.0	72	1.1737
No log	19.0	76	1.1303
No log	20.0	80	1.0276
No log	21.0	84	1.0561
No log	22.0	88	1.0313
No log	23.0	92	1.0517
No log	24.0	96	1.0312
No log	25.0	100	1.0145
No log	26.0	104	1.0391
No log	27.0	108	1.0543
No log	28.0	112	1.0626
No log	29.0	116	1.0653
No log	30.0	120	1.0629

Framework versions

• Transformers 4.20.1
• Pytorch 1.10.0+cu113
• Datasets 2.10.1
• Tokenizers 0.12.1