Hugging Face's logo

ibm-granite
/
granite-3b-code-base-128k

Text Generation
Transformers
Safetensors
llama
code
granite
Eval Results
text-generation-inference
Model card Files Community
rpand002 commited on
Commit
6742973
·
verified ·
1 Parent(s): e6ab4e0

update results

Browse files
Files changed (1) hide show
  1. README.md +121 -1
README.md CHANGED
@@ -65,12 +65,132 @@ model-index:
65
  type: text-generation
66
  dataset:
67
  type: repoqa
68
- name: RepoQA@16K
69
  metrics:
70
  - name: pass@1 (thresh=0.5)
71
  type: pass@1 (thresh=0.5)
72
  value: 40.0
73
  verified: false
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
74
  ---
75
 
76
  ![image/png](https://cdn-uploads.huggingface.co/production/uploads/62cd5057674cdb524450093d/1hzxoPwqkBJXshKVVe6_9.png)
 
65
  type: text-generation
66
  dataset:
67
  type: repoqa
68
+ name: RepoQA (Python@16K)
69
  metrics:
70
  - name: pass@1 (thresh=0.5)
71
  type: pass@1 (thresh=0.5)
72
  value: 40.0
73
  verified: false
74
+ - task:
75
+ type: text-generation
76
+ dataset:
77
+ type: repoqa
78
+ name: RepoQA (C++@16K)
79
+ metrics:
80
+ - name: pass@1 (thresh=0.5)
81
+ type: pass@1 (thresh=0.5)
82
+ value: 36.0
83
+ verified: false
84
+ - task:
85
+ type: text-generation
86
+ dataset:
87
+ type: repoqa
88
+ name: RepoQA (Java@16K)
89
+ metrics:
90
+ - name: pass@1 (thresh=0.5)
91
+ type: pass@1 (thresh=0.5)
92
+ value: 37.0
93
+ verified: false
94
+ - task:
95
+ type: text-generation
96
+ dataset:
97
+ type: repoqa
98
+ name: RepoQA (TypeScript@16K)
99
+ metrics:
100
+ - name: pass@1 (thresh=0.5)
101
+ type: pass@1 (thresh=0.5)
102
+ value: 27.0
103
+ verified: false
104
+ - task:
105
+ type: text-generation
106
+ dataset:
107
+ type: repoqa
108
+ name: RepoQA (Rust@16K)
109
+ metrics:
110
+ - name: pass@1 (thresh=0.5)
111
+ type: pass@1 (thresh=0.5)
112
+ value: 39.0
113
+ verified: false
114
+ - task:
115
+ type: text-generation
116
+ dataset:
117
+ type: lcc
118
+ name: LCC(Balanced)
119
+ metrics:
120
+ - name: Exact Match@4K
121
+ type: Exact Match@4K
122
+ value: 54.6
123
+ verified: false
124
+ - task:
125
+ type: text-generation
126
+ dataset:
127
+ type: lcc
128
+ name: LCC(Balanced)
129
+ metrics:
130
+ - name: Exact Match@16K
131
+ type: Exact Match@16K
132
+ value: 56.8
133
+ verified: false
134
+ - task:
135
+ type: text-generation
136
+ dataset:
137
+ type: lcc
138
+ name: LCC(Balanced)
139
+ metrics:
140
+ - name: Exact Match@16K
141
+ type: Exact Match@16K
142
+ value: 52.2
143
+ verified: false
144
+ - task:
145
+ type: text-generation
146
+ dataset:
147
+ type: lcc
148
+ name: LCC(Balanced)
149
+ metrics:
150
+ - name: Exact Match@32K
151
+ type: Exact Match@32K
152
+ value: 57.8
153
+ verified: false
154
+ - task:
155
+ type: text-generation
156
+ dataset:
157
+ type: repobench
158
+ name: RepoBench-P(Balanced)
159
+ metrics:
160
+ - name: Exact Match@4K
161
+ type: Exact Match@4K
162
+ value: 39.8
163
+ verified: false
164
+ - task:
165
+ type: text-generation
166
+ dataset:
167
+ type: repobench
168
+ name: RepoBench-P(Balanced)
169
+ metrics:
170
+ - name: Exact Match@16K
171
+ type: Exact Match@16K
172
+ value: 46.8
173
+ verified: false
174
+ - task:
175
+ type: text-generation
176
+ dataset:
177
+ type: repobench
178
+ name: RepoBench-P(Balanced)
179
+ metrics:
180
+ - name: Exact Match@16K
181
+ type: Exact Match@16K
182
+ value: 43.1
183
+ verified: false
184
+ - task:
185
+ type: text-generation
186
+ dataset:
187
+ type: repobench
188
+ name: RepoBench-P(Balanced)
189
+ metrics:
190
+ - name: Exact Match@32K
191
+ type: Exact Match@32K
192
+ value: 45.3
193
+ verified: false
194
  ---
195
 
196
  ![image/png](https://cdn-uploads.huggingface.co/production/uploads/62cd5057674cdb524450093d/1hzxoPwqkBJXshKVVe6_9.png)