Update README.md
Browse files
README.md
CHANGED
@@ -14,24 +14,16 @@ library_name: transformers
|
|
14 |
|
15 |
Model created by analyzing and selecting the optimal layers from other Qwen2.5-7B models based on their dimensional utilization efficiency, measured by the Normalized Effective Rank (NER). Computed like:
|
16 |
|
17 |
-
|
18 |
-
- Input: Weight matrix A ∈ R^(m×n) # m = number of output features, n = number of input features
|
19 |
- Compute singular values σᵢ where σᵢ ≥ 0 # σᵢ represents the importance of each dimension
|
20 |
-
- Filter values above numerical threshold (>1e-12)
|
21 |
-
|
22 |
-
Distribution Normalization:
|
23 |
- Sum all singular values: S = Σσᵢ # S acts as normalization factor
|
24 |
- Create probability distribution: pᵢ = σᵢ/S # converts singular values to probabilities summing to 1
|
25 |
-
|
26 |
-
|
27 |
-
- Compute Shannon entropy: H = -Σ(pᵢ * log₂(pᵢ)) # measures information content of distribution
|
28 |
-
- Calculate maximum possible entropy: H_max = log₂(n) # n = number of singular values
|
29 |
-
where n is the number of singular values # maximum entropy occurs when all dimensions contribute equally
|
30 |
-
|
31 |
-
Normalization:
|
32 |
- Final NER score = H/H_max # normalizes score to [0,1] range
|
33 |
-
- Results in value between 0 and 1
|
34 |
-
|
35 |
|
36 |
## Creating Composite Model
|
37 |
|
@@ -63,6 +55,8 @@ fine_tuned_models: # uncomment the models you want to merge
|
|
63 |
|
64 |
#- "Qwen/Qwen2.5-7B-Instruct"
|
65 |
|
|
|
|
|
66 |
#- "FourOhFour/Vapor_v2_7B"
|
67 |
|
68 |
#- "Goekdeniz-Guelmez/Josiefied-Qwen2.5-7B-Instruct-abliterated-v2"
|
@@ -77,21 +71,41 @@ fine_tuned_models: # uncomment the models you want to merge
|
|
77 |
|
78 |
#- "Orion-zhen/Meissa-Qwen2.5-7B-Instruct"
|
79 |
|
|
|
|
|
80 |
#- "jeffmeloy/Qwen2.5-7B-nerd-uncensored-v1.0"
|
81 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
82 |
#- "rombodawg/Rombos-LLM-V2.5-Qwen-7b"
|
83 |
|
84 |
#- "Cran-May/T.E-8.1"
|
85 |
|
86 |
-
#- "thomas-yanxin/XinYuan-Qwen2.5-7B-0917"
|
87 |
-
|
88 |
#- "beomi/Qwen2.5-7B-Instruct-kowiki-qa"
|
89 |
|
90 |
#- "Orion-zhen/Qwen2.5-7B-Gutenberg-KTO"
|
91 |
|
92 |
-
#-
|
|
|
|
|
93 |
|
94 |
-
#-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
95 |
|
96 |
#- "Qwen/Qwen2.5-Coder-7B-Instruct"
|
97 |
|
@@ -101,11 +115,13 @@ fine_tuned_models: # uncomment the models you want to merge
|
|
101 |
|
102 |
#- "Qwen/Qwen2.5-Math-7B"
|
103 |
|
104 |
-
#- "
|
105 |
|
106 |
-
#- "
|
107 |
|
108 |
-
#- "
|
|
|
|
|
109 |
|
110 |
models_dir: "./input_models/"
|
111 |
|
|
|
14 |
|
15 |
Model created by analyzing and selecting the optimal layers from other Qwen2.5-7B models based on their dimensional utilization efficiency, measured by the Normalized Effective Rank (NER). Computed like:
|
16 |
|
17 |
+
- Input: Weight matrix for each model layer
|
|
|
18 |
- Compute singular values σᵢ where σᵢ ≥ 0 # σᵢ represents the importance of each dimension
|
19 |
+
- Filter values above numerical threshold (>1e-12)
|
|
|
|
|
20 |
- Sum all singular values: S = Σσᵢ # S acts as normalization factor
|
21 |
- Create probability distribution: pᵢ = σᵢ/S # converts singular values to probabilities summing to 1
|
22 |
+
- Compute Shannon entropy: H = -Σ(pᵢ * log₂(pᵢ)) # measures information content
|
23 |
+
- Calculate maximum possible entropy: H_max = log₂(n)
|
|
|
|
|
|
|
|
|
|
|
24 |
- Final NER score = H/H_max # normalizes score to [0,1] range
|
25 |
+
- Results in value between 0 and 1
|
26 |
+
# 0 = single dimension dominance, 1 = uniform dimensional utilization
|
27 |
|
28 |
## Creating Composite Model
|
29 |
|
|
|
55 |
|
56 |
#- "Qwen/Qwen2.5-7B-Instruct"
|
57 |
|
58 |
+
#- "EVA-UNIT-01/EVA-Qwen2.5-7B-v0.1"
|
59 |
+
|
60 |
#- "FourOhFour/Vapor_v2_7B"
|
61 |
|
62 |
#- "Goekdeniz-Guelmez/Josiefied-Qwen2.5-7B-Instruct-abliterated-v2"
|
|
|
71 |
|
72 |
#- "Orion-zhen/Meissa-Qwen2.5-7B-Instruct"
|
73 |
|
74 |
+
#- "jeffmeloy/Qwen2.5-7B-nerd-uncensored-v0.9"
|
75 |
+
|
76 |
#- "jeffmeloy/Qwen2.5-7B-nerd-uncensored-v1.0"
|
77 |
|
78 |
+
#- "jeffmeloy/Qwen2.5-7B-nerd-uncensored-v1.1"
|
79 |
+
|
80 |
+
#- "jeffmeloy/Qwen2.5-7B-nerd-uncensored-v1.2"
|
81 |
+
|
82 |
+
#- "AmberYifan/Qwen2.5-7B-dpo-2k"
|
83 |
+
|
84 |
+
#- "sethuiyer/Qwen2.5-7B-Anvita"
|
85 |
+
|
86 |
#- "rombodawg/Rombos-LLM-V2.5-Qwen-7b"
|
87 |
|
88 |
#- "Cran-May/T.E-8.1"
|
89 |
|
|
|
|
|
90 |
#- "beomi/Qwen2.5-7B-Instruct-kowiki-qa"
|
91 |
|
92 |
#- "Orion-zhen/Qwen2.5-7B-Gutenberg-KTO"
|
93 |
|
94 |
+
#- "fblgit/cybertron-v4-qw7B-MGS"
|
95 |
+
|
96 |
+
#- "nguyentd/FinancialAdvice-Qwen2.5-7B"
|
97 |
|
98 |
+
#- "WhiteRabbitNeo/WhiteRabbitNeo-2.5-Qwen-2.5-Coder-7B"
|
99 |
+
|
100 |
+
#- "edgerunner-ai/EdgeRunner-Command-Nested"
|
101 |
+
|
102 |
+
#- "katanemo/Arch-Function-7B"
|
103 |
+
|
104 |
+
#- "DeepGlint-AI/llava-mlcd-qwen2.5-7b"
|
105 |
+
|
106 |
+
#- "mergekit-community/mergekit-slerp-aflqaqy"
|
107 |
+
|
108 |
+
#- "mergekit-community/mergekit-ties-inxwsfo"
|
109 |
|
110 |
#- "Qwen/Qwen2.5-Coder-7B-Instruct"
|
111 |
|
|
|
115 |
|
116 |
#- "Qwen/Qwen2.5-Math-7B"
|
117 |
|
118 |
+
#- "thomas-yanxin/XinYuan-Qwen2.5-7B-0917"
|
119 |
|
120 |
+
#- "jbjeong91/Qwen2.5_7B_IST_StoryGen_vanilla"
|
121 |
|
122 |
+
#- "AmberYifan/Qwen2.5-7B-dpo-2k-hhrlhf"
|
123 |
+
|
124 |
+
#- "jbjeong91/Qwen2.5_7B_IST_StoryGen_test2"
|
125 |
|
126 |
models_dir: "./input_models/"
|
127 |
|