Hugging Face's logo

lesso18
/
3b18b30e-4587-4407-af19-2483e9f3eb56

PEFT
Safetensors
llama
axolotl
Generated from Trainer
Model card Files Community
3b18b30e-4587-4407-af19-2483e9f3eb56 / README.md
lesso18's picture
lesso18
End of training
57e8787 verified
preview code
|
raw
history blame contribute delete
2.22 kB
metadata
library_name: peft
base_model: Casual-Autopsy/L3-Umbral-Mind-RP-v3.0-8B
tags:
  - axolotl
  - generated_from_trainer
model-index:
  - name: 3b18b30e-4587-4407-af19-2483e9f3eb56
    results: []

Built with Axolotl

3b18b30e-4587-4407-af19-2483e9f3eb56

This model is a fine-tuned version of Casual-Autopsy/L3-Umbral-Mind-RP-v3.0-8B on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 1.3190

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.000218
  • train_batch_size: 4
  • eval_batch_size: 4
  • seed: 42
  • gradient_accumulation_steps: 2
  • total_train_batch_size: 8
  • optimizer: Use OptimizerNames.ADAMW_BNB with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
  • lr_scheduler_type: cosine
  • lr_scheduler_warmup_steps: 50
  • training_steps: 402

Training results

Training Loss Epoch Step Validation Loss
No log 0.0025 1 2.3575
1.3761 0.1245 50 1.7705
1.0668 0.2491 100 1.6715
1.0146 0.3736 150 1.5669
0.9796 0.4981 200 1.5292
0.9781 0.6227 250 1.4715
0.8543 0.7472 300 1.4179
1.0009 0.8717 350 1.3170
0.9256 0.9963 400 1.3190

Framework versions

  • PEFT 0.13.2
  • Transformers 4.46.0
  • Pytorch 2.5.0+cu124
  • Datasets 3.0.1
  • Tokenizers 0.20.1