Hugging Face's logo

lesso17
/
e09c1903-89ed-4406-8155-01eadf3b8d77

PEFT
Safetensors
llama
axolotl
Generated from Trainer
Model card Files Community
e09c1903-89ed-4406-8155-01eadf3b8d77 / README.md
lesso17's picture
lesso17
End of training
f7553e2 verified
preview code
|
raw
history blame contribute delete
2.32 kB
metadata
library_name: peft
base_model: Casual-Autopsy/L3-Umbral-Mind-RP-v3.0-8B
tags:
  - axolotl
  - generated_from_trainer
model-index:
  - name: e09c1903-89ed-4406-8155-01eadf3b8d77
    results: []

Built with Axolotl

e09c1903-89ed-4406-8155-01eadf3b8d77

This model is a fine-tuned version of Casual-Autopsy/L3-Umbral-Mind-RP-v3.0-8B on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 0.9177

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.000217
  • train_batch_size: 4
  • eval_batch_size: 4
  • seed: 42
  • gradient_accumulation_steps: 2
  • total_train_batch_size: 8
  • optimizer: Use OptimizerNames.ADAMW_BNB with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
  • lr_scheduler_type: cosine
  • lr_scheduler_warmup_steps: 50
  • training_steps: 500

Training results

Training Loss Epoch Step Validation Loss
No log 0.0004 1 1.6687
1.0486 0.0213 50 1.2570
0.9823 0.0425 100 1.1912
1.1053 0.0638 150 1.2409
1.1105 0.0850 200 1.1848
0.9607 0.1063 250 1.0670
0.9153 0.1276 300 1.0058
1.016 0.1488 350 0.9650
1.1709 0.1701 400 0.9385
1.0365 0.1913 450 0.9181
0.8909 0.2126 500 0.9177

Framework versions

  • PEFT 0.13.2
  • Transformers 4.46.0
  • Pytorch 2.5.0+cu124
  • Datasets 3.0.1
  • Tokenizers 0.20.1