Training in progress, step 1500, checkpoint

last-checkpoint/README.md

@@ -0,0 +1,202 @@
+---
+base_model: jhflow/mistral7b-lora-multi-turn-v2
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.2

last-checkpoint/adapter_config.json

@@ -0,0 +1,161 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "jhflow/mistral7b-lora-multi-turn-v2",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.3,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": [
+    "lm_head"
+  ],
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "model.layers.7.mlp.down_proj",
+    "model.layers.18.self_attn.v_proj",
+    "model.layers.17.self_attn.v_proj",
+    "model.layers.11.self_attn.o_proj",
+    "model.layers.31.mlp.gate_proj",
+    "model.layers.9.mlp.gate_proj",
+    "model.layers.16.mlp.down_proj",
+    "model.layers.27.mlp.down_proj",
+    "model.layers.2.mlp.up_proj",
+    "model.layers.1.mlp.down_proj",
+    "model.layers.24.mlp.down_proj",
+    "model.layers.16.self_attn.q_proj",
+    "model.layers.2.mlp.down_proj",
+    "model.layers.4.self_attn.q_proj",
+    "model.layers.20.mlp.up_proj",
+    "model.layers.11.self_attn.q_proj",
+    "model.layers.16.self_attn.o_proj",
+    "model.layers.30.self_attn.q_proj",
+    "model.layers.8.self_attn.k_proj",
+    "model.layers.29.self_attn.k_proj",
+    "model.layers.14.mlp.down_proj",
+    "model.layers.4.self_attn.k_proj",
+    "model.layers.2.self_attn.q_proj",
+    "model.layers.12.mlp.gate_proj",
+    "model.layers.31.mlp.down_proj",
+    "model.layers.22.mlp.up_proj",
+    "model.layers.5.mlp.gate_proj",
+    "model.layers.14.mlp.gate_proj",
+    "model.layers.1.mlp.up_proj",
+    "model.layers.20.self_attn.k_proj",
+    "model.layers.23.self_attn.v_proj",
+    "model.layers.3.self_attn.q_proj",
+    "model.layers.6.self_attn.v_proj",
+    "model.embed_tokens",
+    "model.layers.14.self_attn.q_proj",
+    "model.layers.0.mlp.down_proj",
+    "model.layers.7.self_attn.q_proj",
+    "model.layers.5.mlp.down_proj",
+    "model.layers.17.self_attn.o_proj",
+    "model.layers.9.mlp.down_proj",
+    "model.layers.16.self_attn.k_proj",
+    "model.layers.13.mlp.down_proj",
+    "model.layers.13.mlp.up_proj",
+    "model.layers.18.self_attn.k_proj",
+    "model.layers.1.self_attn.k_proj",
+    "model.layers.31.self_attn.q_proj",
+    "model.layers.10.mlp.gate_proj",
+    "model.layers.28.mlp.down_proj",
+    "model.layers.6.mlp.gate_proj",
+    "model.layers.27.self_attn.v_proj",
+    "model.layers.31.self_attn.o_proj",
+    "model.layers.19.mlp.down_proj",
+    "model.layers.10.self_attn.o_proj",
+    "model.layers.1.self_attn.q_proj",
+    "model.layers.8.mlp.down_proj",
+    "model.layers.11.self_attn.v_proj",
+    "model.layers.30.mlp.up_proj",
+    "model.layers.18.mlp.down_proj",
+    "model.layers.17.mlp.down_proj",
+    "model.layers.29.self_attn.v_proj",
+    "model.layers.7.self_attn.o_proj",
+    "model.layers.13.self_attn.v_proj",
+    "model.layers.23.self_attn.q_proj",
+    "model.layers.0.self_attn.v_proj",
+    "model.layers.19.self_attn.q_proj",
+    "model.layers.19.mlp.up_proj",
+    "model.layers.22.self_attn.q_proj",
+    "model.layers.28.self_attn.o_proj",
+    "model.layers.24.self_attn.k_proj",
+    "model.layers.5.self_attn.o_proj",
+    "model.layers.21.self_attn.o_proj",
+    "model.layers.31.self_attn.k_proj",
+    "model.layers.13.mlp.gate_proj",
+    "model.layers.7.self_attn.k_proj",
+    "model.layers.6.self_attn.k_proj",
+    "model.layers.11.mlp.gate_proj",
+    "model.layers.3.mlp.down_proj",
+    "model.layers.25.mlp.up_proj",
+    "model.layers.30.self_attn.k_proj",
+    "model.layers.28.mlp.up_proj",
+    "model.layers.10.self_attn.v_proj",
+    "model.layers.6.self_attn.o_proj",
+    "model.layers.14.self_attn.k_proj",
+    "model.layers.26.self_attn.v_proj",
+    "model.layers.12.self_attn.o_proj",
+    "model.layers.23.self_attn.o_proj",
+    "model.layers.3.mlp.gate_proj",
+    "model.layers.19.self_attn.k_proj",
+    "model.layers.28.mlp.gate_proj",
+    "model.layers.18.mlp.up_proj",
+    "model.layers.17.self_attn.k_proj",
+    "model.layers.8.self_attn.v_proj",
+    "model.layers.20.mlp.gate_proj",
+    "model.layers.9.self_attn.v_proj",
+    "model.layers.21.self_attn.v_proj",
+    "model.layers.26.self_attn.o_proj",
+    "model.layers.4.mlp.down_proj",
+    "model.layers.15.mlp.up_proj",
+    "model.layers.0.mlp.up_proj",
+    "model.layers.29.self_attn.o_proj",
+    "model.layers.25.self_attn.k_proj",
+    "model.layers.25.self_attn.o_proj",
+    "model.layers.0.self_attn.q_proj",
+    "model.layers.2.mlp.gate_proj",
+    "model.layers.3.self_attn.o_proj",
+    "model.layers.15.self_attn.v_proj",
+    "model.layers.27.mlp.gate_proj",
+    "model.layers.21.self_attn.k_proj",
+    "model.layers.31.mlp.up_proj",
+    "model.layers.4.mlp.up_proj",
+    "model.layers.20.mlp.down_proj",
+    "model.layers.23.mlp.down_proj",
+    "model.layers.27.self_attn.k_proj",
+    "model.layers.12.mlp.up_proj",
+    "model.layers.22.self_attn.k_proj",
+    "model.layers.30.mlp.down_proj",
+    "model.layers.5.self_attn.k_proj",
+    "model.layers.21.mlp.down_proj",
+    "model.layers.25.mlp.gate_proj",
+    "model.layers.24.self_attn.v_proj",
+    "model.layers.10.self_attn.q_proj",
+    "model.layers.31.self_attn.v_proj",
+    "model.layers.29.mlp.up_proj",
+    "model.layers.26.mlp.up_proj",
+    "model.layers.9.self_attn.q_proj",
+    "model.layers.8.mlp.gate_proj",
+    "model.layers.15.self_attn.k_proj",
+    "model.layers.12.self_attn.q_proj",
+    "model.layers.15.self_attn.q_proj",
+    "model.layers.24.self_attn.q_proj",
+    "model.layers.26.self_attn.k_proj",
+    "model.layers.22.self_attn.v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": true
+}

last-checkpoint/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e5302f6d9a091db432164c1a63f41e30b3cce4a2d92e47aa531eab6d682b1d25
+size 661507488

last-checkpoint/added_tokens.json

@@ -0,0 +1,4 @@
+{
+  "<|im_end|>": 32000,
+  "<|im_start|>": 32001
+}

last-checkpoint/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:91acf3ec2c91626400d32c03c446261a5789dd420a7accad3be4d7540f9c7bc9
+size 1304683322

last-checkpoint/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:37bb2cbf2710152770d3548336dd39456f3f3fd9772b1a23c01e3a79e5e83a9e
+size 14244

last-checkpoint/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c72ff83b81e4d5ca4579296bd0d644ce6acb0bdb379683b8988ce492e8fcf1fd
+size 1256

last-checkpoint/special_tokens_map.json

@@ -0,0 +1,30 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": true,
+    "normalized": false,
+    "rstrip": true,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<unk>",
+    "lstrip": true,
+    "normalized": false,
+    "rstrip": true,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": true,
+    "normalized": false,
+    "rstrip": true,
+    "single_word": false
+  }
+}

last-checkpoint/tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dadfd56d766715c61d2ef780a525ab43b8e6da4de6865bda3d95fdef5e134055
+size 493443

last-checkpoint/tokenizer_config.json

@@ -0,0 +1,68 @@
+{
+  "add_bos_token": true,
+  "add_eos_token": false,
+  "add_prefix_space": null,
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<unk>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<s>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "32000": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "32001": {
+      "content": "<|im_start|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [],
+  "bos_token": "<s>",
+  "chat_template": "{% if not add_generation_prompt is defined %}{% set add_generation_prompt = false %}{% endif %}{% for message in messages %}{{'<|im_start|>' + message['role'] + '\n' + message['content'] + '<|im_end|>' + '\n'}}{% endfor %}{% if add_generation_prompt %}{{ '<|im_start|>assistant\n' }}{% endif %}",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "extra_special_tokens": {},
+  "legacy": true,
+  "max_length": 4096,
+  "model_max_length": 32768,
+  "pad_token": "<unk>",
+  "padding_side": "left",
+  "sp_model_kwargs": {},
+  "spaces_between_special_tokens": false,
+  "stride": 0,
+  "tokenizer_class": "LlamaTokenizer",
+  "truncation_side": "right",
+  "truncation_strategy": "longest_first",
+  "trust_remote_code": false,
+  "unk_token": "<unk>",
+  "use_default_system_prompt": false,
+  "use_fast": true
+}

last-checkpoint/trainer_state.json

@@ -0,0 +1,1163 @@
+{
+  "best_metric": 1.4356995820999146,
+  "best_model_checkpoint": "./output/checkpoint-1500",
+  "epoch": 0.04015849218248019,
+  "eval_steps": 150,
+  "global_step": 1500,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.0002677232812165346,
+      "grad_norm": 12.16030502319336,
+      "learning_rate": 4.4e-06,
+      "loss": 1.5442,
+      "step": 10
+    },
+    {
+      "epoch": 0.0005354465624330692,
+      "grad_norm": 12.028721809387207,
+      "learning_rate": 8.8e-06,
+      "loss": 1.5341,
+      "step": 20
+    },
+    {
+      "epoch": 0.0008031698436496038,
+      "grad_norm": 9.741438865661621,
+      "learning_rate": 1.3199999999999999e-05,
+      "loss": 1.5258,
+      "step": 30
+    },
+    {
+      "epoch": 0.0010708931248661383,
+      "grad_norm": 11.791377067565918,
+      "learning_rate": 1.76e-05,
+      "loss": 1.4766,
+      "step": 40
+    },
+    {
+      "epoch": 0.0013386164060826729,
+      "grad_norm": 10.31489372253418,
+      "learning_rate": 2.2e-05,
+      "loss": 1.4898,
+      "step": 50
+    },
+    {
+      "epoch": 0.0016063396872992076,
+      "grad_norm": 11.65046501159668,
+      "learning_rate": 2.6399999999999998e-05,
+      "loss": 1.4536,
+      "step": 60
+    },
+    {
+      "epoch": 0.0018740629685157421,
+      "grad_norm": 11.001107215881348,
+      "learning_rate": 3.0799999999999996e-05,
+      "loss": 1.4933,
+      "step": 70
+    },
+    {
+      "epoch": 0.0021417862497322766,
+      "grad_norm": 10.670427322387695,
+      "learning_rate": 3.52e-05,
+      "loss": 1.4816,
+      "step": 80
+    },
+    {
+      "epoch": 0.002409509530948811,
+      "grad_norm": 11.35387134552002,
+      "learning_rate": 3.96e-05,
+      "loss": 1.4636,
+      "step": 90
+    },
+    {
+      "epoch": 0.0026772328121653457,
+      "grad_norm": 10.275943756103516,
+      "learning_rate": 4.4e-05,
+      "loss": 1.4996,
+      "step": 100
+    },
+    {
+      "epoch": 0.0029449560933818807,
+      "grad_norm": 9.64588451385498,
+      "learning_rate": 4.399954783308405e-05,
+      "loss": 1.5114,
+      "step": 110
+    },
+    {
+      "epoch": 0.003212679374598415,
+      "grad_norm": 9.566597938537598,
+      "learning_rate": 4.399819135092302e-05,
+      "loss": 1.542,
+      "step": 120
+    },
+    {
+      "epoch": 0.0034804026558149497,
+      "grad_norm": 10.304100036621094,
+      "learning_rate": 4.399593060927658e-05,
+      "loss": 1.4911,
+      "step": 130
+    },
+    {
+      "epoch": 0.0037481259370314842,
+      "grad_norm": 8.622906684875488,
+      "learning_rate": 4.3992765701074955e-05,
+      "loss": 1.4655,
+      "step": 140
+    },
+    {
+      "epoch": 0.004015849218248019,
+      "grad_norm": 9.855925559997559,
+      "learning_rate": 4.398869675641513e-05,
+      "loss": 1.5424,
+      "step": 150
+    },
+    {
+      "epoch": 0.004015849218248019,
+      "eval_loss": 1.5061633586883545,
+      "eval_runtime": 76.7585,
+      "eval_samples_per_second": 6.514,
+      "eval_steps_per_second": 6.514,
+      "step": 150
+    },
+    {
+      "epoch": 0.004283572499464553,
+      "grad_norm": 9.257708549499512,
+      "learning_rate": 4.398372394255549e-05,
+      "loss": 1.4863,
+      "step": 160
+    },
+    {
+      "epoch": 0.004551295780681088,
+      "grad_norm": 7.029283046722412,
+      "learning_rate": 4.397784746390892e-05,
+      "loss": 1.504,
+      "step": 170
+    },
+    {
+      "epoch": 0.004819019061897622,
+      "grad_norm": 8.059552192687988,
+      "learning_rate": 4.3971067562034454e-05,
+      "loss": 1.4734,
+      "step": 180
+    },
+    {
+      "epoch": 0.005086742343114157,
+      "grad_norm": 8.756830215454102,
+      "learning_rate": 4.39633845156273e-05,
+      "loss": 1.5308,
+      "step": 190
+    },
+    {
+      "epoch": 0.005354465624330691,
+      "grad_norm": 8.60611629486084,
+      "learning_rate": 4.39547986405074e-05,
+      "loss": 1.4985,
+      "step": 200
+    },
+    {
+      "epoch": 0.005622188905547227,
+      "grad_norm": 8.613848686218262,
+      "learning_rate": 4.3945310289606455e-05,
+      "loss": 1.493,
+      "step": 210
+    },
+    {
+      "epoch": 0.005889912186763761,
+      "grad_norm": 7.788877010345459,
+      "learning_rate": 4.39349198529534e-05,
+      "loss": 1.492,
+      "step": 220
+    },
+    {
+      "epoch": 0.006157635467980296,
+      "grad_norm": 7.323453903198242,
+      "learning_rate": 4.39236277576584e-05,
+      "loss": 1.5138,
+      "step": 230
+    },
+    {
+      "epoch": 0.00642535874919683,
+      "grad_norm": 7.350474834442139,
+      "learning_rate": 4.391143446789526e-05,
+      "loss": 1.5314,
+      "step": 240
+    },
+    {
+      "epoch": 0.006693082030413365,
+      "grad_norm": 7.383894443511963,
+      "learning_rate": 4.389834048488236e-05,
+      "loss": 1.5231,
+      "step": 250
+    },
+    {
+      "epoch": 0.006960805311629899,
+      "grad_norm": 7.436572551727295,
+      "learning_rate": 4.388434634686206e-05,
+      "loss": 1.5264,
+      "step": 260
+    },
+    {
+      "epoch": 0.007228528592846434,
+      "grad_norm": 7.9078779220581055,
+      "learning_rate": 4.386945262907856e-05,
+      "loss": 1.4744,
+      "step": 270
+    },
+    {
+      "epoch": 0.0074962518740629685,
+      "grad_norm": 7.032011985778809,
+      "learning_rate": 4.3853659943754275e-05,
+      "loss": 1.4156,
+      "step": 280
+    },
+    {
+      "epoch": 0.007763975155279503,
+      "grad_norm": 7.743986129760742,
+      "learning_rate": 4.383696894006463e-05,
+      "loss": 1.5338,
+      "step": 290
+    },
+    {
+      "epoch": 0.008031698436496038,
+      "grad_norm": 7.996994495391846,
+      "learning_rate": 4.381938030411141e-05,
+      "loss": 1.4912,
+      "step": 300
+    },
+    {
+      "epoch": 0.008031698436496038,
+      "eval_loss": 1.5058677196502686,
+      "eval_runtime": 76.63,
+      "eval_samples_per_second": 6.525,
+      "eval_steps_per_second": 6.525,
+      "step": 300
+    },
+    {
+      "epoch": 0.008299421717712573,
+      "grad_norm": 6.838953971862793,
+      "learning_rate": 4.380089475889457e-05,
+      "loss": 1.4854,
+      "step": 310
+    },
+    {
+      "epoch": 0.008567144998929107,
+      "grad_norm": 7.260242462158203,
+      "learning_rate": 4.378151306428244e-05,
+      "loss": 1.5401,
+      "step": 320
+    },
+    {
+      "epoch": 0.008834868280145642,
+      "grad_norm": 6.737756729125977,
+      "learning_rate": 4.3761236016980594e-05,
+      "loss": 1.5013,
+      "step": 330
+    },
+    {
+      "epoch": 0.009102591561362176,
+      "grad_norm": 6.963688373565674,
+      "learning_rate": 4.3740064450499026e-05,
+      "loss": 1.4989,
+      "step": 340
+    },
+    {
+      "epoch": 0.009370314842578711,
+      "grad_norm": 7.6938557624816895,
+      "learning_rate": 4.37179992351179e-05,
+      "loss": 1.5317,
+      "step": 350
+    },
+    {
+      "epoch": 0.009638038123795245,
+      "grad_norm": 6.700031757354736,
+      "learning_rate": 4.3695041277851804e-05,
+      "loss": 1.405,
+      "step": 360
+    },
+    {
+      "epoch": 0.00990576140501178,
+      "grad_norm": 6.925078868865967,
+      "learning_rate": 4.367119152241245e-05,
+      "loss": 1.4966,
+      "step": 370
+    },
+    {
+      "epoch": 0.010173484686228314,
+      "grad_norm": 6.87849235534668,
+      "learning_rate": 4.364645094916985e-05,
+      "loss": 1.4933,
+      "step": 380
+    },
+    {
+      "epoch": 0.01044120796744485,
+      "grad_norm": 8.074585914611816,
+      "learning_rate": 4.3620820575112083e-05,
+      "loss": 1.4782,
+      "step": 390
+    },
+    {
+      "epoch": 0.010708931248661383,
+      "grad_norm": 7.07144832611084,
+      "learning_rate": 4.359430145380344e-05,
+      "loss": 1.4871,
+      "step": 400
+    },
+    {
+      "epoch": 0.010976654529877918,
+      "grad_norm": 7.2088117599487305,
+      "learning_rate": 4.356689467534112e-05,
+      "loss": 1.4855,
+      "step": 410
+    },
+    {
+      "epoch": 0.011244377811094454,
+      "grad_norm": 7.868666648864746,
+      "learning_rate": 4.353860136631044e-05,
+      "loss": 1.5246,
+      "step": 420
+    },
+    {
+      "epoch": 0.011512101092310987,
+      "grad_norm": 7.853616237640381,
+      "learning_rate": 4.350942268973854e-05,
+      "loss": 1.5302,
+      "step": 430
+    },
+    {
+      "epoch": 0.011779824373527523,
+      "grad_norm": 7.353667259216309,
+      "learning_rate": 4.347935984504649e-05,
+      "loss": 1.4305,
+      "step": 440
+    },
+    {
+      "epoch": 0.012047547654744056,
+      "grad_norm": 6.460302352905273,
+      "learning_rate": 4.344841406800012e-05,
+      "loss": 1.4506,
+      "step": 450
+    },
+    {
+      "epoch": 0.012047547654744056,
+      "eval_loss": 1.4909805059432983,
+      "eval_runtime": 76.606,
+      "eval_samples_per_second": 6.527,
+      "eval_steps_per_second": 6.527,
+      "step": 450
+    },
+    {
+      "epoch": 0.012315270935960592,
+      "grad_norm": 7.3225226402282715,
+      "learning_rate": 4.34165866306591e-05,
+      "loss": 1.461,
+      "step": 460
+    },
+    {
+      "epoch": 0.012582994217177125,
+      "grad_norm": 6.5725297927856445,
+      "learning_rate": 4.3383878841324734e-05,
+      "loss": 1.4007,
+      "step": 470
+    },
+    {
+      "epoch": 0.01285071749839366,
+      "grad_norm": 7.1205315589904785,
+      "learning_rate": 4.3350292044486125e-05,
+      "loss": 1.557,
+      "step": 480
+    },
+    {
+      "epoch": 0.013118440779610194,
+      "grad_norm": 6.783862113952637,
+      "learning_rate": 4.331582762076494e-05,
+      "loss": 1.5214,
+      "step": 490
+    },
+    {
+      "epoch": 0.01338616406082673,
+      "grad_norm": 6.475296974182129,
+      "learning_rate": 4.328048698685865e-05,
+      "loss": 1.4874,
+      "step": 500
+    },
+    {
+      "epoch": 0.013653887342043263,
+      "grad_norm": 7.158287525177002,
+      "learning_rate": 4.32442715954823e-05,
+      "loss": 1.4794,
+      "step": 510
+    },
+    {
+      "epoch": 0.013921610623259799,
+      "grad_norm": 5.706000804901123,
+      "learning_rate": 4.320718293530877e-05,
+      "loss": 1.4921,
+      "step": 520
+    },
+    {
+      "epoch": 0.014189333904476333,
+      "grad_norm": 7.483499050140381,
+      "learning_rate": 4.3169222530907634e-05,
+      "loss": 1.4899,
+      "step": 530
+    },
+    {
+      "epoch": 0.014457057185692868,
+      "grad_norm": 6.9520182609558105,
+      "learning_rate": 4.313039194268243e-05,
+      "loss": 1.4908,
+      "step": 540
+    },
+    {
+      "epoch": 0.014724780466909402,
+      "grad_norm": 6.7435784339904785,
+      "learning_rate": 4.309069276680653e-05,
+      "loss": 1.45,
+      "step": 550
+    },
+    {
+      "epoch": 0.014992503748125937,
+      "grad_norm": 7.162035942077637,
+      "learning_rate": 4.305012663515759e-05,
+      "loss": 1.4702,
+      "step": 560
+    },
+    {
+      "epoch": 0.015260227029342472,
+      "grad_norm": 6.717561721801758,
+      "learning_rate": 4.300869521525039e-05,
+      "loss": 1.5131,
+      "step": 570
+    },
+    {
+      "epoch": 0.015527950310559006,
+      "grad_norm": 6.205082893371582,
+      "learning_rate": 4.296640021016832e-05,
+      "loss": 1.4342,
+      "step": 580
+    },
+    {
+      "epoch": 0.01579567359177554,
+      "grad_norm": 6.673497676849365,
+      "learning_rate": 4.292324335849338e-05,
+      "loss": 1.4917,
+      "step": 590
+    },
+    {
+      "epoch": 0.016063396872992075,
+      "grad_norm": 6.291605472564697,
+      "learning_rate": 4.287922643423471e-05,
+      "loss": 1.5018,
+      "step": 600
+    },
+    {
+      "epoch": 0.016063396872992075,
+      "eval_loss": 1.476717472076416,
+      "eval_runtime": 76.7435,
+      "eval_samples_per_second": 6.515,
+      "eval_steps_per_second": 6.515,
+      "step": 600
+    },
+    {
+      "epoch": 0.01633112015420861,
+      "grad_norm": 6.723529815673828,
+      "learning_rate": 4.283435124675567e-05,
+      "loss": 1.4652,
+      "step": 610
+    },
+    {
+      "epoch": 0.016598843435425146,
+      "grad_norm": 6.2483062744140625,
+      "learning_rate": 4.278861964069944e-05,
+      "loss": 1.5094,
+      "step": 620
+    },
+    {
+      "epoch": 0.01686656671664168,
+      "grad_norm": 6.773886203765869,
+      "learning_rate": 4.274203349591324e-05,
+      "loss": 1.4771,
+      "step": 630
+    },
+    {
+      "epoch": 0.017134289997858213,
+      "grad_norm": 7.28003454208374,
+      "learning_rate": 4.269459472737102e-05,
+      "loss": 1.436,
+      "step": 640
+    },
+    {
+      "epoch": 0.017402013279074747,
+      "grad_norm": 6.813667297363281,
+      "learning_rate": 4.264630528509473e-05,
+      "loss": 1.4094,
+      "step": 650
+    },
+    {
+      "epoch": 0.017669736560291284,
+      "grad_norm": 7.017402172088623,
+      "learning_rate": 4.259716715407422e-05,
+      "loss": 1.5255,
+      "step": 660
+    },
+    {
+      "epoch": 0.017937459841507818,
+      "grad_norm": 7.313465595245361,
+      "learning_rate": 4.254718235418559e-05,
+      "loss": 1.4647,
+      "step": 670
+    },
+    {
+      "epoch": 0.01820518312272435,
+      "grad_norm": 6.323640823364258,
+      "learning_rate": 4.249635294010819e-05,
+      "loss": 1.4799,
+      "step": 680
+    },
+    {
+      "epoch": 0.01847290640394089,
+      "grad_norm": 7.1620588302612305,
+      "learning_rate": 4.244468100124014e-05,
+      "loss": 1.4344,
+      "step": 690
+    },
+    {
+      "epoch": 0.018740629685157422,
+      "grad_norm": 6.160943508148193,
+      "learning_rate": 4.239216866161248e-05,
+      "loss": 1.516,
+      "step": 700
+    },
+    {
+      "epoch": 0.019008352966373956,
+      "grad_norm": 6.571516036987305,
+      "learning_rate": 4.233881807980179e-05,
+      "loss": 1.5133,
+      "step": 710
+    },
+    {
+      "epoch": 0.01927607624759049,
+      "grad_norm": 5.696547031402588,
+      "learning_rate": 4.228463144884155e-05,
+      "loss": 1.4318,
+      "step": 720
+    },
+    {
+      "epoch": 0.019543799528807027,
+      "grad_norm": 6.653096675872803,
+      "learning_rate": 4.2229610996131915e-05,
+      "loss": 1.461,
+      "step": 730
+    },
+    {
+      "epoch": 0.01981152281002356,
+      "grad_norm": 6.497095584869385,
+      "learning_rate": 4.217375898334819e-05,
+      "loss": 1.4359,
+      "step": 740
+    },
+    {
+      "epoch": 0.020079246091240094,
+      "grad_norm": 6.566861629486084,
+      "learning_rate": 4.211707770634788e-05,
+      "loss": 1.445,
+      "step": 750
+    },
+    {
+      "epoch": 0.020079246091240094,
+      "eval_loss": 1.4696097373962402,
+      "eval_runtime": 76.6203,
+      "eval_samples_per_second": 6.526,
+      "eval_steps_per_second": 6.526,
+      "step": 750
+    },
+    {
+      "epoch": 0.020346969372456628,
+      "grad_norm": 6.802274703979492,
+      "learning_rate": 4.205956949507625e-05,
+      "loss": 1.4485,
+      "step": 760
+    },
+    {
+      "epoch": 0.020614692653673165,
+      "grad_norm": 6.832027912139893,
+      "learning_rate": 4.200123671347065e-05,
+      "loss": 1.5034,
+      "step": 770
+    },
+    {
+      "epoch": 0.0208824159348897,
+      "grad_norm": 7.041072368621826,
+      "learning_rate": 4.1942081759363236e-05,
+      "loss": 1.5225,
+      "step": 780
+    },
+    {
+      "epoch": 0.021150139216106232,
+      "grad_norm": 6.773481369018555,
+      "learning_rate": 4.1882107064382496e-05,
+      "loss": 1.4718,
+      "step": 790
+    },
+    {
+      "epoch": 0.021417862497322766,
+      "grad_norm": 6.902709007263184,
+      "learning_rate": 4.1821315093853216e-05,
+      "loss": 1.4562,
+      "step": 800
+    },
+    {
+      "epoch": 0.021685585778539303,
+      "grad_norm": 5.957550048828125,
+      "learning_rate": 4.1759708346695215e-05,
+      "loss": 1.4798,
+      "step": 810
+    },
+    {
+      "epoch": 0.021953309059755836,
+      "grad_norm": 6.139000415802002,
+      "learning_rate": 4.1697289355320565e-05,
+      "loss": 1.5084,
+      "step": 820
+    },
+    {
+      "epoch": 0.02222103234097237,
+      "grad_norm": 6.227492332458496,
+      "learning_rate": 4.1634060685529527e-05,
+      "loss": 1.4597,
+      "step": 830
+    },
+    {
+      "epoch": 0.022488755622188907,
+      "grad_norm": 6.531513214111328,
+      "learning_rate": 4.157002493640506e-05,
+      "loss": 1.4326,
+      "step": 840
+    },
+    {
+      "epoch": 0.02275647890340544,
+      "grad_norm": 6.589105606079102,
+      "learning_rate": 4.1505184740206006e-05,
+      "loss": 1.431,
+      "step": 850
+    },
+    {
+      "epoch": 0.023024202184621975,
+      "grad_norm": 6.289806842803955,
+      "learning_rate": 4.143954276225886e-05,
+      "loss": 1.5167,
+      "step": 860
+    },
+    {
+      "epoch": 0.023291925465838508,
+      "grad_norm": 6.444394111633301,
+      "learning_rate": 4.1373101700848235e-05,
+      "loss": 1.4948,
+      "step": 870
+    },
+    {
+      "epoch": 0.023559648747055045,
+      "grad_norm": 6.666561126708984,
+      "learning_rate": 4.1305864287105946e-05,
+      "loss": 1.4879,
+      "step": 880
+    },
+    {
+      "epoch": 0.02382737202827158,
+      "grad_norm": 6.169593811035156,
+      "learning_rate": 4.12378332848987e-05,
+      "loss": 1.5557,
+      "step": 890
+    },
+    {
+      "epoch": 0.024095095309488113,
+      "grad_norm": 6.8753743171691895,
+      "learning_rate": 4.116901149071457e-05,
+      "loss": 1.4446,
+      "step": 900
+    },
+    {
+      "epoch": 0.024095095309488113,
+      "eval_loss": 1.4628037214279175,
+      "eval_runtime": 76.661,
+      "eval_samples_per_second": 6.522,
+      "eval_steps_per_second": 6.522,
+      "step": 900
+    },
+    {
+      "epoch": 0.024362818590704646,
+      "grad_norm": 6.948112964630127,
+      "learning_rate": 4.1099401733547925e-05,
+      "loss": 1.4916,
+      "step": 910
+    },
+    {
+      "epoch": 0.024630541871921183,
+      "grad_norm": 6.508751392364502,
+      "learning_rate": 4.102900687478326e-05,
+      "loss": 1.4659,
+      "step": 920
+    },
+    {
+      "epoch": 0.024898265153137717,
+      "grad_norm": 6.732906818389893,
+      "learning_rate": 4.095782980807749e-05,
+      "loss": 1.4834,
+      "step": 930
+    },
+    {
+      "epoch": 0.02516598843435425,
+      "grad_norm": 6.261349678039551,
+      "learning_rate": 4.088587345924105e-05,
+      "loss": 1.4585,
+      "step": 940
+    },
+    {
+      "epoch": 0.025433711715570784,
+      "grad_norm": 5.926994323730469,
+      "learning_rate": 4.081314078611762e-05,
+      "loss": 1.4887,
+      "step": 950
+    },
+    {
+      "epoch": 0.02570143499678732,
+      "grad_norm": 6.746396064758301,
+      "learning_rate": 4.073963477846249e-05,
+      "loss": 1.4561,
+      "step": 960
+    },
+    {
+      "epoch": 0.025969158278003855,
+      "grad_norm": 6.503916263580322,
+      "learning_rate": 4.066535845781975e-05,
+      "loss": 1.5013,
+      "step": 970
+    },
+    {
+      "epoch": 0.02623688155922039,
+      "grad_norm": 6.83921480178833,
+      "learning_rate": 4.059031487739803e-05,
+      "loss": 1.517,
+      "step": 980
+    },
+    {
+      "epoch": 0.026504604840436926,
+      "grad_norm": 6.210860252380371,
+      "learning_rate": 4.051450712194497e-05,
+      "loss": 1.4849,
+      "step": 990
+    },
+    {
+      "epoch": 0.02677232812165346,
+      "grad_norm": 6.381270885467529,
+      "learning_rate": 4.043793830762049e-05,
+      "loss": 1.4685,
+      "step": 1000
+    },
+    {
+      "epoch": 0.027040051402869993,
+      "grad_norm": 6.763075351715088,
+      "learning_rate": 4.036061158186866e-05,
+      "loss": 1.5412,
+      "step": 1010
+    },
+    {
+      "epoch": 0.027307774684086527,
+      "grad_norm": 6.492913722991943,
+      "learning_rate": 4.028253012328828e-05,
+      "loss": 1.4398,
+      "step": 1020
+    },
+    {
+      "epoch": 0.027575497965303064,
+      "grad_norm": 6.383675575256348,
+      "learning_rate": 4.0203697141502323e-05,
+      "loss": 1.4514,
+      "step": 1030
+    },
+    {
+      "epoch": 0.027843221246519598,
+      "grad_norm": 6.685030937194824,
+      "learning_rate": 4.0124115877025874e-05,
+      "loss": 1.4688,
+      "step": 1040
+    },
+    {
+      "epoch": 0.02811094452773613,
+      "grad_norm": 6.681553840637207,
+      "learning_rate": 4.004378960113303e-05,
+      "loss": 1.4862,
+      "step": 1050
+    },
+    {
+      "epoch": 0.02811094452773613,
+      "eval_loss": 1.4577986001968384,
+      "eval_runtime": 76.6839,
+      "eval_samples_per_second": 6.52,
+      "eval_steps_per_second": 6.52,
+      "step": 1050
+    },
+    {
+      "epoch": 0.028378667808952665,
+      "grad_norm": 6.192584037780762,
+      "learning_rate": 3.996272161572237e-05,
+      "loss": 1.4383,
+      "step": 1060
+    },
+    {
+      "epoch": 0.028646391090169202,
+      "grad_norm": 6.383575439453125,
+      "learning_rate": 3.988091525318126e-05,
+      "loss": 1.4015,
+      "step": 1070
+    },
+    {
+      "epoch": 0.028914114371385736,
+      "grad_norm": 7.042336940765381,
+      "learning_rate": 3.979837387624884e-05,
+      "loss": 1.468,
+      "step": 1080
+    },
+    {
+      "epoch": 0.02918183765260227,
+      "grad_norm": 6.60390567779541,
+      "learning_rate": 3.971510087787784e-05,
+      "loss": 1.4932,
+      "step": 1090
+    },
+    {
+      "epoch": 0.029449560933818803,
+      "grad_norm": 6.5373029708862305,
+      "learning_rate": 3.9631099681095044e-05,
+      "loss": 1.4381,
+      "step": 1100
+    },
+    {
+      "epoch": 0.02971728421503534,
+      "grad_norm": 7.118472099304199,
+      "learning_rate": 3.954637373886066e-05,
+      "loss": 1.4057,
+      "step": 1110
+    },
+    {
+      "epoch": 0.029985007496251874,
+      "grad_norm": 7.386786460876465,
+      "learning_rate": 3.9460926533926315e-05,
+      "loss": 1.4978,
+      "step": 1120
+    },
+    {
+      "epoch": 0.030252730777468408,
+      "grad_norm": 6.418981552124023,
+      "learning_rate": 3.937476157869193e-05,
+      "loss": 1.4897,
+      "step": 1130
+    },
+    {
+      "epoch": 0.030520454058684945,
+      "grad_norm": 5.941694259643555,
+      "learning_rate": 3.9287882415061334e-05,
+      "loss": 1.4381,
+      "step": 1140
+    },
+    {
+      "epoch": 0.03078817733990148,
+      "grad_norm": 6.574525833129883,
+      "learning_rate": 3.9200292614296655e-05,
+      "loss": 1.4143,
+      "step": 1150
+    },
+    {
+      "epoch": 0.031055900621118012,
+      "grad_norm": 6.349545478820801,
+      "learning_rate": 3.911199577687154e-05,
+      "loss": 1.3937,
+      "step": 1160
+    },
+    {
+      "epoch": 0.03132362390233455,
+      "grad_norm": 6.873767375946045,
+      "learning_rate": 3.902299553232315e-05,
+      "loss": 1.4515,
+      "step": 1170
+    },
+    {
+      "epoch": 0.03159134718355108,
+      "grad_norm": 6.043056964874268,
+      "learning_rate": 3.893329553910293e-05,
+      "loss": 1.5538,
+      "step": 1180
+    },
+    {
+      "epoch": 0.031859070464767617,
+      "grad_norm": 6.418127059936523,
+      "learning_rate": 3.884289948442628e-05,
+      "loss": 1.4745,
+      "step": 1190
+    },
+    {
+      "epoch": 0.03212679374598415,
+      "grad_norm": 6.102353096008301,
+      "learning_rate": 3.875181108412096e-05,
+      "loss": 1.4772,
+      "step": 1200
+    },
+    {
+      "epoch": 0.03212679374598415,
+      "eval_loss": 1.449093222618103,
+      "eval_runtime": 76.8079,
+      "eval_samples_per_second": 6.51,
+      "eval_steps_per_second": 6.51,
+      "step": 1200
+    },
+    {
+      "epoch": 0.032394517027200684,
+      "grad_norm": 6.624678134918213,
+      "learning_rate": 3.8660034082474316e-05,
+      "loss": 1.4526,
+      "step": 1210
+    },
+    {
+      "epoch": 0.03266224030841722,
+      "grad_norm": 6.601352214813232,
+      "learning_rate": 3.856757225207944e-05,
+      "loss": 1.5247,
+      "step": 1220
+    },
+    {
+      "epoch": 0.03292996358963375,
+      "grad_norm": 6.580589771270752,
+      "learning_rate": 3.847442939368002e-05,
+      "loss": 1.4694,
+      "step": 1230
+    },
+    {
+      "epoch": 0.03319768687085029,
+      "grad_norm": 5.504952430725098,
+      "learning_rate": 3.8380609336014156e-05,
+      "loss": 1.411,
+      "step": 1240
+    },
+    {
+      "epoch": 0.033465410152066825,
+      "grad_norm": 6.643352508544922,
+      "learning_rate": 3.828611593565694e-05,
+      "loss": 1.4278,
+      "step": 1250
+    },
+    {
+      "epoch": 0.03373313343328336,
+      "grad_norm": 6.425754070281982,
+      "learning_rate": 3.819095307686197e-05,
+      "loss": 1.4253,
+      "step": 1260
+    },
+    {
+      "epoch": 0.03400085671449989,
+      "grad_norm": 6.0909504890441895,
+      "learning_rate": 3.809512467140163e-05,
+      "loss": 1.4681,
+      "step": 1270
+    },
+    {
+      "epoch": 0.034268579995716426,
+      "grad_norm": 5.82462215423584,
+      "learning_rate": 3.799863465840634e-05,
+      "loss": 1.4275,
+      "step": 1280
+    },
+    {
+      "epoch": 0.03453630327693296,
+      "grad_norm": 6.24934720993042,
+      "learning_rate": 3.790148700420261e-05,
+      "loss": 1.4313,
+      "step": 1290
+    },
+    {
+      "epoch": 0.034804026558149494,
+      "grad_norm": 6.211289882659912,
+      "learning_rate": 3.7803685702150006e-05,
+      "loss": 1.4216,
+      "step": 1300
+    },
+    {
+      "epoch": 0.035071749839366034,
+      "grad_norm": 5.66799259185791,
+      "learning_rate": 3.7705234772476984e-05,
+      "loss": 1.4799,
+      "step": 1310
+    },
+    {
+      "epoch": 0.03533947312058257,
+      "grad_norm": 6.351191997528076,
+      "learning_rate": 3.760613826211567e-05,
+      "loss": 1.4281,
+      "step": 1320
+    },
+    {
+      "epoch": 0.0356071964017991,
+      "grad_norm": 5.707607269287109,
+      "learning_rate": 3.7506400244535455e-05,
+      "loss": 1.463,
+      "step": 1330
+    },
+    {
+      "epoch": 0.035874919683015635,
+      "grad_norm": 5.780487537384033,
+      "learning_rate": 3.740602481957561e-05,
+      "loss": 1.4731,
+      "step": 1340
+    },
+    {
+      "epoch": 0.03614264296423217,
+      "grad_norm": 5.354376316070557,
+      "learning_rate": 3.7305016113276704e-05,
+      "loss": 1.4492,
+      "step": 1350
+    },
+    {
+      "epoch": 0.03614264296423217,
+      "eval_loss": 1.444738745689392,
+      "eval_runtime": 76.8239,
+      "eval_samples_per_second": 6.508,
+      "eval_steps_per_second": 6.508,
+      "step": 1350
+    },
+    {
+      "epoch": 0.0364103662454487,
+      "grad_norm": 5.93463134765625,
+      "learning_rate": 3.7203378277711024e-05,
+      "loss": 1.4602,
+      "step": 1360
+    },
+    {
+      "epoch": 0.036678089526665236,
+      "grad_norm": 5.922112464904785,
+      "learning_rate": 3.710111549081191e-05,
+      "loss": 1.4412,
+      "step": 1370
+    },
+    {
+      "epoch": 0.03694581280788178,
+      "grad_norm": 6.667977809906006,
+      "learning_rate": 3.699823195620199e-05,
+      "loss": 1.4475,
+      "step": 1380
+    },
+    {
+      "epoch": 0.03721353608909831,
+      "grad_norm": 6.021790504455566,
+      "learning_rate": 3.689473190302041e-05,
+      "loss": 1.4206,
+      "step": 1390
+    },
+    {
+      "epoch": 0.037481259370314844,
+      "grad_norm": 6.152276039123535,
+      "learning_rate": 3.679061958574897e-05,
+      "loss": 1.4288,
+      "step": 1400
+    },
+    {
+      "epoch": 0.03774898265153138,
+      "grad_norm": 5.695444583892822,
+      "learning_rate": 3.668589928403726e-05,
+      "loss": 1.4424,
+      "step": 1410
+    },
+    {
+      "epoch": 0.03801670593274791,
+      "grad_norm": 6.346884727478027,
+      "learning_rate": 3.6580575302526706e-05,
+      "loss": 1.5001,
+      "step": 1420
+    },
+    {
+      "epoch": 0.038284429213964445,
+      "grad_norm": 5.674633979797363,
+      "learning_rate": 3.647465197067368e-05,
+      "loss": 1.4796,
+      "step": 1430
+    },
+    {
+      "epoch": 0.03855215249518098,
+      "grad_norm": 6.168262481689453,
+      "learning_rate": 3.6368133642571464e-05,
+      "loss": 1.4428,
+      "step": 1440
+    },
+    {
+      "epoch": 0.03881987577639751,
+      "grad_norm": 6.4981369972229,
+      "learning_rate": 3.6261024696771345e-05,
+      "loss": 1.4281,
+      "step": 1450
+    },
+    {
+      "epoch": 0.03908759905761405,
+      "grad_norm": 5.703588962554932,
+      "learning_rate": 3.615332953610255e-05,
+      "loss": 1.3934,
+      "step": 1460
+    },
+    {
+      "epoch": 0.03935532233883059,
+      "grad_norm": 5.69433069229126,
+      "learning_rate": 3.604505258749132e-05,
+      "loss": 1.4482,
+      "step": 1470
+    },
+    {
+      "epoch": 0.03962304562004712,
+      "grad_norm": 5.763268947601318,
+      "learning_rate": 3.5936198301778945e-05,
+      "loss": 1.4629,
+      "step": 1480
+    },
+    {
+      "epoch": 0.039890768901263654,
+      "grad_norm": 6.472227096557617,
+      "learning_rate": 3.5826771153538716e-05,
+      "loss": 1.4301,
+      "step": 1490
+    },
+    {
+      "epoch": 0.04015849218248019,
+      "grad_norm": 6.155264854431152,
+      "learning_rate": 3.571677564089214e-05,
+      "loss": 1.4703,
+      "step": 1500
+    },
+    {
+      "epoch": 0.04015849218248019,
+      "eval_loss": 1.4356995820999146,
+      "eval_runtime": 76.8148,
+      "eval_samples_per_second": 6.509,
+      "eval_steps_per_second": 6.509,
+      "step": 1500
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 5000,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 1,
+  "save_steps": 150,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 3.401789867004672e+17,
+  "train_batch_size": 4,
+  "trial_name": null,
+  "trial_params": null
+}

last-checkpoint/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5183a7bec814dfa8bac416983643755ef1307dfd795a1dc3ecfe479de9ef2406
+size 5496