Update README.md
Browse files
README.md
CHANGED
|
@@ -10,6 +10,7 @@ license: cc-by-nc-sa-4.0
|
|
| 10 |
- nGPT
|
| 11 |
- ResFormer
|
| 12 |
- NeuTRENO (as in resformer)
|
|
|
|
| 13 |
|
| 14 |
## Architecture:
|
| 15 |
- 32 Layers
|
|
|
|
| 10 |
- nGPT
|
| 11 |
- ResFormer
|
| 12 |
- NeuTRENO (as in resformer)
|
| 13 |
+
- Tanh logit softcapping (as in Gemma2)
|
| 14 |
|
| 15 |
## Architecture:
|
| 16 |
- 32 Layers
|