Upload folder using huggingface_hub
Browse files
README.md
CHANGED
|
@@ -6,7 +6,7 @@ pipeline_tag: text-generation
|
|
| 6 |
|
| 7 |
# Intro
|
| 8 |
|
| 9 |
-
[Activation Beacon](https://arxiv.org/abs/2401.03462) compresses the original KV into fewer yet more compact states (a.k.a. beacons) and hence
|
| 10 |
- **Effective**
|
| 11 |
- there is little information loss given a compression ratio of 2, 4, and 8;
|
| 12 |
- **Efficient**
|
|
|
|
| 6 |
|
| 7 |
# Intro
|
| 8 |
|
| 9 |
+
[Activation Beacon](https://arxiv.org/abs/2401.03462) compresses the original KV into fewer yet more compact states (a.k.a. beacons) and hence enables the LLM to perceive longer context given its fixed context window. It is known for the following features:
|
| 10 |
- **Effective**
|
| 11 |
- there is little information loss given a compression ratio of 2, 4, and 8;
|
| 12 |
- **Efficient**
|