Update README.md
Browse files
README.md
CHANGED
@@ -1,9 +1,24 @@
|
|
1 |
---
|
|
|
|
|
|
|
2 |
tags:
|
3 |
-
- model_hub_mixin
|
4 |
-
- pytorch_model_hub_mixin
|
|
|
|
|
5 |
---
|
6 |
|
7 |
-
|
8 |
-
|
9 |
-
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
---
|
2 |
+
language: en
|
3 |
+
library_name: mlsae
|
4 |
+
license: mit
|
5 |
tags:
|
6 |
+
- model_hub_mixin
|
7 |
+
- pytorch_model_hub_mixin
|
8 |
+
datasets:
|
9 |
+
- monology/pile-uncopyrighted
|
10 |
---
|
11 |
|
12 |
+
# mlsae-gpt2-x64-k32
|
13 |
+
|
14 |
+
A Multi-Layer Sparse Autoencoder (MLSAE) trained on the residual stream
|
15 |
+
activation vectors from every layer of
|
16 |
+
[openai-community/gpt2](https://huggingface.co/openai-community/gpt2)
|
17 |
+
with an expansion factor of 64 and k = 32, over 1 billion tokens from
|
18 |
+
[monology/pile-uncopyrighted](https://huggingface.co/datasets/monology/pile-uncopyrighted).
|
19 |
+
|
20 |
+
For more details, see:
|
21 |
+
|
22 |
+
- Paper: <https://arxiv.org/abs/2409.04185>
|
23 |
+
- GitHub repository: <https://github.com/tim-lawson/mlsae>
|
24 |
+
- Weights & Biases project: <https://wandb.ai/timlawson-/mlsae>
|