OpenMOSE
/

PRWKV-7-Phi-4-Instruct-Preview-v0.1

Model card Files Files and versions Community

OpenMOSE commited on Mar 14

Commit

59f8bb2

·

1 Parent(s): 4e10654

Update README.md

Files changed (1) hide show

README.md +2 -1

README.md CHANGED Viewed

@@ -47,7 +47,8 @@ The training process consisted of three distinct stages:
 ### Stage 3: Supervised Fine-Tuning (Using RWKV-LM-RLHF)
 - Utilized a distillation dataset of 900K samples
-- Implemented VariableRankPEFT to enhance training efficiency
 ## How to Use
 - PC Requirements 16GB+ VRAM NVIDIA GPU(rocm also can use. but only fp16.)

 ### Stage 3: Supervised Fine-Tuning (Using RWKV-LM-RLHF)
 - Utilized a distillation dataset of 900K samples
+- Implemented Variable Rank PEFT to enhance training efficiency
+- Bone(Block Affine Transformation) r=512+
 ## How to Use
 - PC Requirements 16GB+ VRAM NVIDIA GPU(rocm also can use. but only fp16.)