Lambent commited on
Commit
dcf52e6
·
verified ·
1 Parent(s): 4f91585

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +5 -0
README.md CHANGED
@@ -13,6 +13,11 @@ tags:
13
  This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
14
 
15
  ## Merge Details
 
 
 
 
 
16
  ### Merge Method
17
 
18
  This model was merged using the SLERP merge method.
 
13
  This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
14
 
15
  ## Merge Details
16
+
17
+ Same idea as Lambent/qwen2.5-14B-selfmerge-A, but training the base model on an ~20M token instruct and continued pretraining dataset first.
18
+
19
+ Hope is the lightweight instruction tuning might add some synergy with the original instruct.
20
+
21
  ### Merge Method
22
 
23
  This model was merged using the SLERP merge method.