Update README.md

README.md

@@ -14,7 +14,7 @@ pipeline_tag: question-answering
 
 # **Doge 20M Instruct**
 
-![architecture](https://cdn-uploads.huggingface.co/production/uploads/673ab3647afcea17eb4378fd/BBPbIRiw1KxdgYp_5CABf.png)
+![architecture](Doge.png)
 
 Doge is an ongoing research project where we aim to train a series of small language models to further explore whether the Transformer framework allows for more complex feedforward network structures, enabling the model to have fewer cache states and larger knowledge capacity.
 
@@ -70,8 +70,8 @@ We build the Doge-Instruct by first SFT on [SmolTalk](https://huggingface.co/dat
 **SFT**:
 | Model | Training Data | Epochs | Content Length | LR | Batch Size | Precision |
 |---|---|---|---|---|---|---|
-| [Doge-20M-Instruct-SFT](https://huggingface.co/JingzeShi/Doge-20M-Instruct-SFT) | [HuggingFaceTB/smoltalk](https://huggingface.co/datasets/HuggingFaceTB/smoltalk) | 2 | 8192 | 8e-4 | 1M | bfloat16 |
-| [Doge-60M-Instruct](https://huggingface.co/JingzeShi/Doge-60M-Instruct) | [HuggingFaceTB/smoltalk](https://huggingface.co/datasets/HuggingFaceTB/smoltalk) | 2 | 8192 | 6e-4 | 1M | bfloat16 |
+| [Doge-20M-Instruct-SFT](https://huggingface.co/JingzeShi/Doge-20M-Instruct-SFT) | [HuggingFaceTB/smoltalk](https://huggingface.co/datasets/HuggingFaceTB/smoltalk) | 2 | 2048 | 8e-4 | 0.25M | bfloat16 |
+| [Doge-60M-Instruct](https://huggingface.co/JingzeShi/Doge-60M-Instruct) | [HuggingFaceTB/smoltalk](https://huggingface.co/datasets/HuggingFaceTB/smoltalk) | 2 | 2048 | 6e-4 | 0.25M | bfloat16 |
 
 **DPO**:
 | Model | Training Data | Epochs | Content Length | LR | Batch Size | Precision |
@@ -83,10 +83,10 @@ We build the Doge-Instruct by first SFT on [SmolTalk](https://huggingface.co/dat
 **Procedure**:
 
 **SFT**:
-[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="150" height="24"/>](https://wandb.ai/loser_cheems/huggingface/runs/xl21ytg8) 
+[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="150" height="24"/>](https://wandb.ai/loser_cheems/huggingface/runs/eohr6fuj) 
 
 **DPO**:
-[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="150" height="24"/>](https://wandb.ai/loser_cheems/huggingface/runs/874wshj2)
+[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="150" height="24"/>](https://wandb.ai/loser_cheems/huggingface/runs/h6c2p2fe)
 
 
 **Environment**: