htlou/backup_0202_llamafactory_Compcap_l0_0_90-llava-mistral

Compcap_l0_0_90

This model is a fine-tuned version of llava-hf/llava-v1.6-mistral-7b-hf on the Compcap_l0_0_90 dataset. It achieves the following results on the evaluation set:

• Loss: 0.7734

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-06
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• distributed_type: multi-GPU
• num_devices: 8
• gradient_accumulation_steps: 4
• total_train_batch_size: 256
• total_eval_batch_size: 64
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: cosine
• lr_scheduler_warmup_steps: 10
• num_epochs: 3.0

Training results

Training Loss	Epoch	Step	Validation Loss
0.9397	0.1505	50	0.9223
0.9027	0.3010	100	0.8662
0.8711	0.4515	150	0.8410
0.8628	0.6020	200	0.8240
0.8283	0.7524	250	0.8112
0.7989	0.9029	300	0.8007
0.7415	1.0534	350	0.7968
0.7347	1.2039	400	0.7904
0.7181	1.3544	450	0.7858
0.7435	1.5049	500	0.7818
0.7384	1.6554	550	0.7777
0.719	1.8059	600	0.7739
0.7179	1.9564	650	0.7713
0.6837	2.1068	700	0.7764
0.6684	2.2573	750	0.7755
0.6869	2.4078	800	0.7743
0.6717	2.5583	850	0.7738
0.6793	2.7088	900	0.7734
0.6685	2.8593	950	0.7734

Framework versions

• Transformers 4.45.2
• Pytorch 2.4.0+cu121
• Datasets 2.21.0
• Tokenizers 0.20.3