Update README.md
Browse files
README.md
CHANGED
@@ -7,14 +7,14 @@ license: apache-2.0
|
|
7 |
|
8 |
## Model Overview
|
9 |
|
10 |
-
PRWKV-7-Phi-4-Instruct is a large language model based on the RNN-based RWKV-x070 architecture, comprising 16.
|
11 |
|
12 |
## Technical Specifications
|
13 |
|
14 |
- **Architecture**: RWKV-x070 "Goose"(RNN-based) https://github.com/BlinkDL/RWKV-LM
|
15 |
-
- **Parameters**: 16.
|
16 |
- **Training Context Window**: 12288(Stage1=2560, Stage2=8192, Stage3=12288)
|
17 |
-
- **Base Model**: Derived from Microsoft Phi-4 14B
|
18 |
- **Development Stage**: Experimental preview (no performance guarantees)
|
19 |
- **License**: Apache 2.0
|
20 |
|
|
|
7 |
|
8 |
## Model Overview
|
9 |
|
10 |
+
PRWKV-7-Phi-4-Instruct is a large language model based on the RNN-based RWKV-x070 architecture, comprising 16.3 billion parameters. The distinctive feature of this model is that it replaces the attention mechanism in Microsoft's Transformer-based Phi-4 14B with RWKV's recurrent approach.
|
11 |
|
12 |
## Technical Specifications
|
13 |
|
14 |
- **Architecture**: RWKV-x070 "Goose"(RNN-based) https://github.com/BlinkDL/RWKV-LM
|
15 |
+
- **Parameters**: 16.3 billion
|
16 |
- **Training Context Window**: 12288(Stage1=2560, Stage2=8192, Stage3=12288)
|
17 |
+
- **Base Model**: Derived from Microsoft Phi-4 14B https://huggingface.co/microsoft/phi-4
|
18 |
- **Development Stage**: Experimental preview (no performance guarantees)
|
19 |
- **License**: Apache 2.0
|
20 |
|