Update README.md

README.md

@@ -2,7 +2,6 @@
 license: bigscience-bloom-rail-1.0
 datasets:
 - c4
-- CarperAI/pile-v2-small-filtered
 language:
 - en
 library_name: transformers
@@ -20,6 +19,4 @@ rotary_dim - 64
 tokenizer - gpt-j
 ```
 
-Trained on 4,194,304 samples from the [c4](https://hf.co/datasets/c4) dataset, at a length of 128 tokens each, that comes out to 536,870,912 (0.53B) tokens seen during training. A batch size of 16 with 128 gradient accumulation steps was used, making the effective batch size 2048. A cosine learning rate schedule was used starting at 1e-3.
-
-Another 20,480 samples from [CarperAI/pile-v2-small-filtered](https://hf.co/CarperAI/pile-v2-small-filtered) were used for finetuning, again at 128 tokens each (for a total of 2.6M more tokens) at a batch size of 16 with 256 gradient accumulation steps, with a learning rate of 1e-4 with a linearly decreasing schedule. (That's 5 entire steps for y'all counting at home, it took ~4:30 lol)
+Trained on 4,194,304 samples from the [c4](https://hf.co/datasets/c4) dataset, at a length of 128 tokens each, that comes out to 536,870,912 (0.53B) tokens seen during training. A batch size of 16 with 128 gradient accumulation steps was used, making the effective batch size 2048. A cosine learning rate schedule was used starting at 1e-3.