Hugging Face's logo

gliner-community
/
gliner_small-v2.5

Token Classification
GLiNER
PyTorch
multilingual
Model card Files Community
Ihor commited on
Commit
b29764c
·
verified ·
1 Parent(s): a810bbe

add results for other datasets

Browse files
Files changed (1) hide show
  1. README.md +193 -0
README.md CHANGED
@@ -63,6 +63,199 @@ European Championship => competitions
63
  Below is a comparison of results between previous versions of the model and the current one:
64
  ![Models performance](models_comparison.png)
65
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
66
  ## Other available models
67
 
68
  | Release | Model Name | # of Parameters | Language | License |
 
63
  Below is a comparison of results between previous versions of the model and the current one:
64
  ![Models performance](models_comparison.png)
65
 
66
+ ### Results on other datasets
67
+ | Model | Dataset | Precision | Recall | F1 Score |
68
+ |------------------------------------|---------------------|-----------|--------|----------|
69
+ | gliner-community/gliner_small-v2.5 | ACE 2004 | 35.18% | 22.81% | 27.67% |
70
+ | | ACE 2005 | 35.89% | 22.39% | 27.58% |
71
+ | | AnatEM | 49.12% | 31.31% | 38.24% |
72
+ | | Broad Tweet Corpus | 59.51% | 77.85% | 67.46% |
73
+ | | CoNLL 2003 | 63.16% | 70.43% | 66.60% |
74
+ | | FabNER | 23.78% | 22.55% | 23.15% |
75
+ | | FindVehicle | 37.46% | 40.06% | 38.72% |
76
+ | | GENIA_NER | 45.90% | 54.11% | 49.67% |
77
+ | | HarveyNER | 13.20% | 32.58% | 18.78% |
78
+ | | MultiNERD | 45.87% | 87.01% | 60.07% |
79
+ | | Ontonotes | 23.05% | 41.16% | 29.55% |
80
+ | | PolyglotNER | 31.88% | 67.22% | 43.25% |
81
+ | | TweetNER7 | 40.98% | 39.91% | 40.44% |
82
+ | | WikiANN en | 55.35% | 60.06% | 57.61% |
83
+ | | WikiNeural | 64.52% | 86.24% | 73.81% |
84
+ | | bc2gm | 51.70% | 49.99% | 50.83% |
85
+ | | bc4chemd | 30.78% | 57.56% | 40.11% |
86
+ | | bc5cdr | 63.48% | 69.65% | 66.42% |
87
+ | | ncbi | 63.36% | 66.67% | 64.97% |
88
+ | | **Average** | | | **46.58%** |
89
+ |------------------------------------|---------------------|-----------|--------|----------|
90
+ | urchade/gliner_small-v2.1 | ACE 2004 | 38.89% | 23.53% | 29.32% |
91
+ | | ACE 2005 | 42.09% | 26.82% | 32.76% |
92
+ | | AnatEM | 63.71% | 19.45% | 29.80% |
93
+ | | Broad Tweet Corpus | 57.01% | 70.49% | 63.04% |
94
+ | | CoNLL 2003 | 57.11% | 62.66% | 59.76% |
95
+ | | FabNER | 32.41% | 12.33% | 17.87% |
96
+ | | FindVehicle | 43.47% | 33.02% | 37.53% |
97
+ | | GENIA_NER | 61.03% | 37.25% | 46.26% |
98
+ | | HarveyNER | 23.12% | 15.16% | 18.32% |
99
+ | | MultiNERD | 43.63% | 83.60% | 57.34% |
100
+ | | Ontonotes | 23.25% | 35.41% | 28.07% |
101
+ | | PolyglotNER | 29.47% | 64.41% | 40.44% |
102
+ | | TweetNER7 | 44.78% | 30.83% | 36.52% |
103
+ | | WikiANN en | 52.58% | 58.31% | 55.30% |
104
+ | | WikiNeural | 53.38% | 82.19% | 64.72% |
105
+ | | bc2gm | 66.64% | 30.56% | 41.90% |
106
+ | | bc4chemd | 42.01% | 56.03% | 48.02% |
107
+ | | bc5cdr | 72.03% | 58.58% | 64.61% |
108
+ | | ncbi | 68.88% | 46.71% | 55.67% |
109
+ | | **Average** | | | **43.54%** |
110
+ |------------------------------------|---------------------|-----------|--------|----------|
111
+ | EmergentMethods/gliner_small-v2.1 | ACE 2004 | 39.92% | 17.50% | 24.34% |
112
+ | | ACE 2005 | 38.53% | 16.58% | 23.18% |
113
+ | | AnatEM | 55.95% | 25.69% | 35.22% |
114
+ | | Broad Tweet Corpus | 66.63% | 72.00% | 69.21% |
115
+ | | CoNLL 2003 | 62.89% | 58.96% | 60.86% |
116
+ | | FabNER | 32.76% | 13.33% | 18.95% |
117
+ | | FindVehicle | 42.93% | 43.20% | 43.06% |
118
+ | | GENIA_NER | 51.28% | 43.75% | 47.22% |
119
+ | | HarveyNER | 24.82% | 21.52% | 23.05% |
120
+ | | MultiNERD | 59.27% | 80.69% | 68.34% |
121
+ | | Ontonotes | 32.97% | 37.59% | 35.13% |
122
+ | | PolyglotNER | 33.60% | 63.30% | 43.90% |
123
+ | | TweetNER7 | 46.90% | 28.66% | 35.58% |
124
+ | | WikiANN en | 51.91% | 55.43% | 53.61% |
125
+ | | WikiNeural | 70.65% | 82.21% | 75.99% |
126
+ | | bc2gm | 49.95% | 43.13% | 46.29% |
127
+ | | bc4chemd | 35.88% | 71.64% | 47.81% |
128
+ | | bc5cdr | 68.41% | 68.90% | 68.65% |
129
+ | | ncbi | 55.31% | 59.87% | 57.50% |
130
+ | | **Average** | | | **46.20%** |
131
+ |-----------------------------------------|---------------------|-----------|--------|----------|
132
+ | gliner-community/gliner_medium-v2.5 | ACE 2004 | 33.06% | 20.96% | 25.66% |
133
+ | | ACE 2005 | 33.65% | 19.65% | 24.81% |
134
+ | | AnatEM | 52.03% | 35.28% | 42.05% |
135
+ | | Broad Tweet Corpus | 60.57% | 79.09% | 68.60% |
136
+ | | CoNLL 2003 | 63.80% | 68.31% | 65.98% |
137
+ | | FabNER | 26.20% | 22.26% | 24.07% |
138
+ | | FindVehicle | 41.95% | 40.68% | 41.30% |
139
+ | | GENIA_NER | 51.83% | 62.34% | 56.60% |
140
+ | | HarveyNER | 14.04% | 32.17% | 19.55% |
141
+ | | MultiNERD | 47.63% | 88.78% | 62.00% |
142
+ | | Ontonotes | 21.68% | 38.41% | 27.71% |
143
+ | | PolyglotNER | 32.73% | 68.27% | 44.24% |
144
+ | | TweetNER7 | 40.39% | 37.64% | 38.97% |
145
+ | | WikiANN en | 56.41% | 59.90% | 58.10% |
146
+ | | WikiNeural | 65.61% | 86.28% | 74.54% |
147
+ | | bc2gm | 55.20% | 56.71% | 55.95% |
148
+ | | bc4chemd | 35.94% | 63.67% | 45.94% |
149
+ | | bc5cdr | 63.50% | 70.09% | 66.63% |
150
+ | | ncbi | 62.96% | 68.55% | 65.63% |
151
+ | | **Average** | | | **47.81%** |
152
+ |-----------------------------------------|---------------------|-----------|--------|----------|
153
+ | urchade/gliner_medium-v2.1 | ACE 2004 | 36.33% | 22.74% | 27.97% |
154
+ | | ACE 2005 | 40.49% | 25.46% | 31.27% |
155
+ | | AnatEM | 59.75% | 16.87% | 26.31% |
156
+ | | Broad Tweet Corpus | 60.89% | 67.25% | 63.91% |
157
+ | | CoNLL 2003 | 60.62% | 62.39% | 61.50% |
158
+ | | FabNER | 27.72% | 12.24% | 16.98% |
159
+ | | FindVehicle | 41.55% | 31.31% | 35.71% |
160
+ | | GENIA_NER | 60.86% | 43.93% | 51.03% |
161
+ | | HarveyNER | 23.20% | 23.16% | 23.18% |
162
+ | | MultiNERD | 41.25% | 83.74% | 55.27% |
163
+ | | Ontonotes | 20.58% | 34.11% | 25.67% |
164
+ | | PolyglotNER | 31.32% | 64.22% | 42.11% |
165
+ | | TweetNER7 | 44.52% | 33.42% | 38.18% |
166
+ | | WikiANN en | 54.57% | 56.47% | 55.51% |
167
+ | | WikiNeural | 57.60% | 81.57% | 67.52% |
168
+ | | bc2gm | 67.98% | 33.45% | 44.84% |
169
+ | | bc4chemd | 45.66% | 52.00% | 48.62% |
170
+ | | bc5cdr | 72.20% | 58.12% | 64.40% |
171
+ | | ncbi | 73.12% | 49.74% | 59.20% |
172
+ | | **Average** | | | **44.17%** |
173
+ |-----------------------------------------|---------------------|-----------|--------|----------|
174
+ | EmergentMethods/gliner_news_medium-v2.1 | ACE 2004 | 39.21% | 17.24% | 23.95% |
175
+ | | ACE 2005 | 39.82% | 16.48% | 23.31% |
176
+ | | AnatEM | 57.67% | 23.57% | 33.46% |
177
+ | | Broad Tweet Corpus | 69.52% | 65.94% | 67.69% |
178
+ | | CoNLL 2003 | 68.26% | 58.45% | 62.97% |
179
+ | | FabNER | 30.74% | 15.51% | 20.62% |
180
+ | | FindVehicle | 40.33% | 37.37% | 38.79% |
181
+ | | GENIA_NER | 53.70% | 47.73% | 50.54% |
182
+ | | HarveyNER | 26.29% | 27.05% | 26.67% |
183
+ | | MultiNERD | 56.78% | 81.96% | 67.08% |
184
+ | | Ontonotes | 30.90% | 35.86% | 33.19% |
185
+ | | PolyglotNER | 35.98% | 60.96% | 45.25% |
186
+ | | TweetNER7 | 52.37% | 30.50% | 38.55% |
187
+ | | WikiANN en | 53.81% | 52.29% | 53.04% |
188
+ | | WikiNeural | 76.84% | 78.92% | 77.86% |
189
+ | | bc2gm | 62.97% | 44.24% | 51.96% |
190
+ | | bc4chemd | 44.90% | 65.56% | 53.30% |
191
+ | | bc5cdr | 73.93% | 67.03% | 70.31% |
192
+ | | ncbi | 69.53% | 60.82% | 64.88% |
193
+ | | **Average** | | | **47.55%** |
194
+ |-----------------------------------------|---------------------|-----------|--------|----------|
195
+ | gliner-community/gliner_large-v2.5 | ACE 2004 | 31.64% | 22.81% | 26.51% |
196
+ | | ACE 2005 | 32.10% | 22.56% | 26.49% |
197
+ | | AnatEM | 53.64% | 27.82% | 36.64% |
198
+ | | Broad Tweet Corpus | 61.93% | 76.85% | 68.59% |
199
+ | | CoNLL 2003 | 62.83% | 67.71% | 65.18% |
200
+ | | FabNER | 24.54% | 27.03% | 25.73% |
201
+ | | FindVehicle | 40.71% | 56.24% | 47.23% |
202
+ | | GENIA_NER | 43.56% | 52.56% | 47.64% |
203
+ | | HarveyNER | 14.85% | 27.05% | 19.17% |
204
+ | | MultiNERD | 38.04% | 89.17% | 53.33% |
205
+ | | Ontonotes | 17.28% | 40.16% | 24.16% |
206
+ | | PolyglotNER | 32.88% | 63.31% | 43.28% |
207
+ | | TweetNER7 | 38.03% | 41.43% | 39.66% |
208
+ | | WikiANN en | 57.80% | 60.54% | 59.14% |
209
+ | | WikiNeural | 67.72% | 83.94% | 74.96% |
210
+ | | bc2gm | 54.74% | 48.54% | 51.45% |
211
+ | | bc4chemd | 40.20% | 58.66% | 47.71% |
212
+ | | bc5cdr | 66.27% | 71.95% | 69.00% |
213
+ | | ncbi | 68.09% | 61.55% | 64.65% |
214
+ | | **Average** | | | **46.87%** |
215
+ |-----------------------------------------|---------------------|-----------|--------|----------|
216
+ | urchade/gliner_large-v2.1 | ACE 2004 | 37.52% | 25.38% | 30.28% |
217
+ | | ACE 2005 | 39.02% | 29.00% | 33.27% |
218
+ | | AnatEM | 52.86% | 13.64% | 21.68% |
219
+ | | Broad Tweet Corpus | 51.44% | 71.73% | 59.91% |
220
+ | | CoNLL 2003 | 54.86% | 64.98% | 59.49% |
221
+ | | FabNER | 23.98% | 16.00% | 19.19% |
222
+ | | FindVehicle | 47.04% | 57.53% | 51.76% |
223
+ | | GENIA_NER | 58.10% | 49.98% | 53.74% |
224
+ | | HarveyNER | 16.29% | 21.93% | 18.69% |
225
+ | | MultiNERD | 34.09% | 85.43% | 48.74% |
226
+ | | Ontonotes | 14.02% | 32.01% | 19.50% |
227
+ | | PolyglotNER | 28.53% | 64.92% | 39.64% |
228
+ | | TweetNER7 | 38.00% | 34.34% | 36.08% |
229
+ | | WikiANN en | 51.69% | 59.92% | 55.50% |
230
+ | | WikiNeural | 50.94% | 82.08% | 62.87% |
231
+ | | bc2gm | 64.48% | 32.47% | 43.19% |
232
+ | | bc4chemd | 48.66% | 57.52% | 52.72% |
233
+ | | bc5cdr | 72.19% | 64.27% | 68.00% |
234
+ | | ncbi | 69.54% | 52.25% | 59.67% |
235
+ | | **Average** | | | **43.89%** |
236
+ |-----------------------------------------|---------------------|-----------|--------|----------|
237
+ | EmergenMethods/fliner_news_large-v2.1 | ACE 2004 | 43.19% | 18.39% | 25.80% |
238
+ | | ACE 2005 | 45.24% | 21.20% | 28.87% |
239
+ | | AnatEM | 61.51% | 21.66% | 32.04% |
240
+ | | Broad Tweet Corpus | 69.38% | 68.99% | 69.18% |
241
+ | | CoNLL 2003 | 61.47% | 52.18% | 56.45% |
242
+ | | FabNER | 27.42% | 19.11% | 22.52% |
243
+ | | FindVehicle | 46.30% | 62.48% | 53.19% |
244
+ | | GENIA_NER | 54.13% | 54.02% | 54.07% |
245
+ | | HarveyNER | 15.91% | 15.78% | 15.84% |
246
+ | | MultiNERD | 53.73% | 79.07% | 63.98% |
247
+ | | Ontonotes | 26.78% | 39.77% | 32.01% |
248
+ | | PolyglotNER | 34.28% | 55.87% | 42.49% |
249
+ | | TweetNER7 | 48.06% | 28.18% | 35.53% |
250
+ | | WikiANN en | 53.66% | 51.34% | 52.47% |
251
+ | | WikiNeural | 69.81% | 70.75% | 70.28% |
252
+ | | bc2gm | 59.83% | 37.62% | 46.20% |
253
+ | | bc4chemd | 46.24% | 69.15% | 55.42% |
254
+ | | bc5cdr | 71.94% | 70.37% | 71.15% |
255
+ | | ncbi | 70.17% | 61.44% | 65.52% |
256
+ | | **Average** | | | **47.00%** |
257
+ |-----------------------------------------|---------------------|-----------|--------|----------|
258
+
259
  ## Other available models
260
 
261
  | Release | Model Name | # of Parameters | Language | License |