htlou/backup_0202_llamafactory_Compcap_cooccur_0_100-llava-mistral

Compcap_cooccur_0_100

This model is a fine-tuned version of llava-hf/llava-v1.6-mistral-7b-hf on the Compcap_cooccur_0_100 dataset. It achieves the following results on the evaluation set:

• Loss: 0.7824

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-06
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• distributed_type: multi-GPU
• num_devices: 8
• gradient_accumulation_steps: 4
• total_train_batch_size: 256
• total_eval_batch_size: 64
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: cosine
• lr_scheduler_warmup_steps: 10
• num_epochs: 3.0

Training results

Training Loss	Epoch	Step	Validation Loss
0.9444	0.1354	50	0.9377
0.8668	0.2708	100	0.8816
0.8436	0.4062	150	0.8559
0.8192	0.5416	200	0.8384
0.8285	0.6770	250	0.8261
0.8132	0.8125	300	0.8165
0.7997	0.9479	350	0.8076
0.7335	1.0833	400	0.8050
0.7466	1.2187	450	0.7997
0.7264	1.3541	500	0.7957
0.7286	1.4895	550	0.7911
0.7251	1.6249	600	0.7876
0.727	1.7603	650	0.7840
0.7277	1.8957	700	0.7811
0.6724	2.0311	750	0.7858
0.6883	2.1666	800	0.7850
0.6709	2.3020	850	0.7840
0.6598	2.4374	900	0.7834
0.674	2.5728	950	0.7830
0.656	2.7082	1000	0.7828
0.6741	2.8436	1050	0.7825
0.6592	2.9790	1100	0.7824

Framework versions

• Transformers 4.45.2
• Pytorch 2.4.0+cu121
• Datasets 2.21.0
• Tokenizers 0.20.3