jeffmeloy commited on
Commit
036c14a
·
verified ·
1 Parent(s): 2b7c469

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +37 -21
README.md CHANGED
@@ -14,24 +14,16 @@ library_name: transformers
14
 
15
  Model created by analyzing and selecting the optimal layers from other Qwen2.5-7B models based on their dimensional utilization efficiency, measured by the Normalized Effective Rank (NER). Computed like:
16
 
17
- Singular Value Decomposition:
18
- - Input: Weight matrix A ∈ R^(m×n) # m = number of output features, n = number of input features
19
  - Compute singular values σᵢ where σᵢ ≥ 0 # σᵢ represents the importance of each dimension
20
- - Filter values above numerical threshold (>1e-12) # removes numerical noise from computation
21
-
22
- Distribution Normalization:
23
  - Sum all singular values: S = Σσᵢ # S acts as normalization factor
24
  - Create probability distribution: pᵢ = σᵢ/S # converts singular values to probabilities summing to 1
25
-
26
- Entropy Calculation:
27
- - Compute Shannon entropy: H = -Σ(pᵢ * log₂(pᵢ)) # measures information content of distribution
28
- - Calculate maximum possible entropy: H_max = log₂(n) # n = number of singular values
29
- where n is the number of singular values # maximum entropy occurs when all dimensions contribute equally
30
-
31
- Normalization:
32
  - Final NER score = H/H_max # normalizes score to [0,1] range
33
- - Results in value between 0 and 1 # 0 = single dimension dominance, 1 = perfect dimensional utilization
34
- - Higher scores indicate more uniform dimensional utilization
35
 
36
  ## Creating Composite Model
37
 
@@ -63,6 +55,8 @@ fine_tuned_models: # uncomment the models you want to merge
63
 
64
  #- "Qwen/Qwen2.5-7B-Instruct"
65
 
 
 
66
  #- "FourOhFour/Vapor_v2_7B"
67
 
68
  #- "Goekdeniz-Guelmez/Josiefied-Qwen2.5-7B-Instruct-abliterated-v2"
@@ -77,21 +71,41 @@ fine_tuned_models: # uncomment the models you want to merge
77
 
78
  #- "Orion-zhen/Meissa-Qwen2.5-7B-Instruct"
79
 
 
 
80
  #- "jeffmeloy/Qwen2.5-7B-nerd-uncensored-v1.0"
81
 
 
 
 
 
 
 
 
 
82
  #- "rombodawg/Rombos-LLM-V2.5-Qwen-7b"
83
 
84
  #- "Cran-May/T.E-8.1"
85
 
86
- #- "thomas-yanxin/XinYuan-Qwen2.5-7B-0917"
87
-
88
  #- "beomi/Qwen2.5-7B-Instruct-kowiki-qa"
89
 
90
  #- "Orion-zhen/Qwen2.5-7B-Gutenberg-KTO"
91
 
92
- #- 'fblgit/cybertron-v4-qw7B-MGS'
 
 
93
 
94
- #- 'nguyentd/FinancialAdvice-Qwen2.5-7B'
 
 
 
 
 
 
 
 
 
 
95
 
96
  #- "Qwen/Qwen2.5-Coder-7B-Instruct"
97
 
@@ -101,11 +115,13 @@ fine_tuned_models: # uncomment the models you want to merge
101
 
102
  #- "Qwen/Qwen2.5-Math-7B"
103
 
104
- #- "WhiteRabbitNeo/WhiteRabbitNeo-2.5-Qwen-2.5-Coder-7B"
105
 
106
- #- "edgerunner-ai/EdgeRunner-Command-Nested"
107
 
108
- #- "katanemo/Arch-Function-7B"
 
 
109
 
110
  models_dir: "./input_models/"
111
 
 
14
 
15
  Model created by analyzing and selecting the optimal layers from other Qwen2.5-7B models based on their dimensional utilization efficiency, measured by the Normalized Effective Rank (NER). Computed like:
16
 
17
+ - Input: Weight matrix for each model layer
 
18
  - Compute singular values σᵢ where σᵢ ≥ 0 # σᵢ represents the importance of each dimension
19
+ - Filter values above numerical threshold (>1e-12)
 
 
20
  - Sum all singular values: S = Σσᵢ # S acts as normalization factor
21
  - Create probability distribution: pᵢ = σᵢ/S # converts singular values to probabilities summing to 1
22
+ - Compute Shannon entropy: H = -Σ(pᵢ * log₂(pᵢ)) # measures information content
23
+ - Calculate maximum possible entropy: H_max = log₂(n)
 
 
 
 
 
24
  - Final NER score = H/H_max # normalizes score to [0,1] range
25
+ - Results in value between 0 and 1
26
+ # 0 = single dimension dominance, 1 = uniform dimensional utilization
27
 
28
  ## Creating Composite Model
29
 
 
55
 
56
  #- "Qwen/Qwen2.5-7B-Instruct"
57
 
58
+ #- "EVA-UNIT-01/EVA-Qwen2.5-7B-v0.1"
59
+
60
  #- "FourOhFour/Vapor_v2_7B"
61
 
62
  #- "Goekdeniz-Guelmez/Josiefied-Qwen2.5-7B-Instruct-abliterated-v2"
 
71
 
72
  #- "Orion-zhen/Meissa-Qwen2.5-7B-Instruct"
73
 
74
+ #- "jeffmeloy/Qwen2.5-7B-nerd-uncensored-v0.9"
75
+
76
  #- "jeffmeloy/Qwen2.5-7B-nerd-uncensored-v1.0"
77
 
78
+ #- "jeffmeloy/Qwen2.5-7B-nerd-uncensored-v1.1"
79
+
80
+ #- "jeffmeloy/Qwen2.5-7B-nerd-uncensored-v1.2"
81
+
82
+ #- "AmberYifan/Qwen2.5-7B-dpo-2k"
83
+
84
+ #- "sethuiyer/Qwen2.5-7B-Anvita"
85
+
86
  #- "rombodawg/Rombos-LLM-V2.5-Qwen-7b"
87
 
88
  #- "Cran-May/T.E-8.1"
89
 
 
 
90
  #- "beomi/Qwen2.5-7B-Instruct-kowiki-qa"
91
 
92
  #- "Orion-zhen/Qwen2.5-7B-Gutenberg-KTO"
93
 
94
+ #- "fblgit/cybertron-v4-qw7B-MGS"
95
+
96
+ #- "nguyentd/FinancialAdvice-Qwen2.5-7B"
97
 
98
+ #- "WhiteRabbitNeo/WhiteRabbitNeo-2.5-Qwen-2.5-Coder-7B"
99
+
100
+ #- "edgerunner-ai/EdgeRunner-Command-Nested"
101
+
102
+ #- "katanemo/Arch-Function-7B"
103
+
104
+ #- "DeepGlint-AI/llava-mlcd-qwen2.5-7b"
105
+
106
+ #- "mergekit-community/mergekit-slerp-aflqaqy"
107
+
108
+ #- "mergekit-community/mergekit-ties-inxwsfo"
109
 
110
  #- "Qwen/Qwen2.5-Coder-7B-Instruct"
111
 
 
115
 
116
  #- "Qwen/Qwen2.5-Math-7B"
117
 
118
+ #- "thomas-yanxin/XinYuan-Qwen2.5-7B-0917"
119
 
120
+ #- "jbjeong91/Qwen2.5_7B_IST_StoryGen_vanilla"
121
 
122
+ #- "AmberYifan/Qwen2.5-7B-dpo-2k-hhrlhf"
123
+
124
+ #- "jbjeong91/Qwen2.5_7B_IST_StoryGen_test2"
125
 
126
  models_dir: "./input_models/"
127