nvl-og/finetuned-ai

README.md

@@ -1,7 +1,7 @@
 ---
 library_name: transformers
-license: llama3.1
-base_model: meta-llama/Meta-Llama-3.1-8B-Instruct
+license: llama3.2
+base_model: meta-llama/Llama-3.2-3b-instruct
 tags:
 - generated_from_trainer
 model-index:
@@ -14,14 +14,9 @@ should probably proofread and complete it, then remove this comment. -->
 
 # results
 
-This model is a fine-tuned version of [meta-llama/Meta-Llama-3.1-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct) on the None dataset.
+This model is a fine-tuned version of [meta-llama/Llama-3.2-3b-instruct](https://huggingface.co/meta-llama/Llama-3.2-3b-instruct) on the None dataset.
 It achieves the following results on the evaluation set:
-- eval_loss: 0.0337
-- eval_runtime: 12.3558
-- eval_samples_per_second: 17.239
-- eval_steps_per_second: 2.185
-- epoch: 4.0
-- step: 214
+- Loss: 0.2300
 
 ## Model description
 
@@ -44,12 +39,23 @@ The following hyperparameters were used during training:
 - train_batch_size: 2
 - eval_batch_size: 8
 - seed: 42
-- gradient_accumulation_steps: 2
-- total_train_batch_size: 4
+- gradient_accumulation_steps: 8
+- total_train_batch_size: 16
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: cosine
 - num_epochs: 5
 
+### Training results
+
+| Training Loss | Epoch  | Step | Validation Loss |
+|:-------------:|:------:|:----:|:---------------:|
+| No log        | 0.9720 | 13   | 0.4116          |
+| 0.819         | 1.9439 | 26   | 0.3048          |
+| 0.3283        | 2.9907 | 40   | 0.2464          |
+| 0.3283        | 3.9626 | 53   | 0.2307          |
+| 0.2244        | 4.8598 | 65   | 0.2300          |
+
+
 ### Framework versions
 
 - Transformers 4.45.1

model-00001-of-00002.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:5b7454a3729bf9f427a11944fbf17a5aba7f77e8014341cabc05d80e4087dd37
+oid sha256:bfa6b14ee09d776a8eb13cb300a067e43dcb96b7fc1fe6803763f22d1d62e019
 size 4965799096

model-00002-of-00002.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:591690a4081655ea35af3343703376f227d6a378221aab73b7ee7477f68908dd
+oid sha256:406a9b2fa21aeaa3215a662771e3a352512546d1ea722466ebf5e47c93919195
 size 1459729952

runs/Sep27_13-13-09_144f728d997d/events.out.tfevents.1727467990.144f728d997d.6579.0

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:f0c420b46740266421d7debe149bd3d78fb9e8ebf03f8ea7341d1a3a4fbf1d9f
-size 5454
+oid sha256:2ae576dc8ad115556611af860dc59516bb0fccb16bf8785f9dda9b25cfe58bd1
+size 7487