Hugging Face's logo

sparse-encoder
/
example-splade-distilbert-base-uncased-quora-duplicates

Feature Extraction
sentence-transformers
Safetensors
English
distilbert
sparse-encoder
sparse
splade
Generated from Trainer
dataset_size:99000
loss:SpladeLoss
loss:SparseMultipleNegativesRankingLoss
loss:FlopsLoss
Eval Results
text-embeddings-inference
Model card Files Community
arthurbresnu HF Staff commited on
Commit
7b505e6
·
verified ·
1 Parent(s): 36420e5

Add new SparseEncoder model

Browse files
Files changed (11) hide show
  1. 1_SpladePooling/config.json +5 -0
  2. README.md +2014 -0
  3. config.json +23 -0
  4. config_sentence_transformers.json +11 -0
  5. model.safetensors +3 -0
  6. modules.json +14 -0
  7. sentence_bert_config.json +4 -0
  8. special_tokens_map.json +7 -0
  9. tokenizer.json +0 -0
  10. tokenizer_config.json +56 -0
  11. vocab.txt +0 -0
1_SpladePooling/config.json ADDED
@@ -0,0 +1,5 @@
 
 
 
 
 
 
1
+ {
2
+ "pooling_strategy": "max",
3
+ "activation_function": "relu",
4
+ "word_embedding_dimension": 30522
5
+ }
README.md ADDED
@@ -0,0 +1,2014 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - en
4
+ license: apache-2.0
5
+ tags:
6
+ - sentence-transformers
7
+ - sparse-encoder
8
+ - sparse
9
+ - splade
10
+ - generated_from_trainer
11
+ - dataset_size:99000
12
+ - loss:SpladeLoss
13
+ - loss:SparseMultipleNegativesRankingLoss
14
+ - loss:FlopsLoss
15
+ base_model: distilbert/distilbert-base-uncased
16
+ widget:
17
+ - text: How do I know if a girl likes me at school?
18
+ - text: What are some five star hotel in Jaipur?
19
+ - text: Is it normal to fantasize your wife having sex with another man?
20
+ - text: What is the Sahara, and how do the average temperatures there compare to the
21
+ ones in the Simpson Desert?
22
+ - text: What are Hillary Clinton's most recognized accomplishments while Secretary
23
+ of State?
24
+ datasets:
25
+ - sentence-transformers/quora-duplicates
26
+ pipeline_tag: feature-extraction
27
+ library_name: sentence-transformers
28
+ metrics:
29
+ - cosine_accuracy
30
+ - cosine_accuracy_threshold
31
+ - cosine_f1
32
+ - cosine_f1_threshold
33
+ - cosine_precision
34
+ - cosine_recall
35
+ - cosine_ap
36
+ - cosine_mcc
37
+ - dot_accuracy
38
+ - dot_accuracy_threshold
39
+ - dot_f1
40
+ - dot_f1_threshold
41
+ - dot_precision
42
+ - dot_recall
43
+ - dot_ap
44
+ - dot_mcc
45
+ - euclidean_accuracy
46
+ - euclidean_accuracy_threshold
47
+ - euclidean_f1
48
+ - euclidean_f1_threshold
49
+ - euclidean_precision
50
+ - euclidean_recall
51
+ - euclidean_ap
52
+ - euclidean_mcc
53
+ - manhattan_accuracy
54
+ - manhattan_accuracy_threshold
55
+ - manhattan_f1
56
+ - manhattan_f1_threshold
57
+ - manhattan_precision
58
+ - manhattan_recall
59
+ - manhattan_ap
60
+ - manhattan_mcc
61
+ - max_accuracy
62
+ - max_accuracy_threshold
63
+ - max_f1
64
+ - max_f1_threshold
65
+ - max_precision
66
+ - max_recall
67
+ - max_ap
68
+ - max_mcc
69
+ - active_dims
70
+ - sparsity_ratio
71
+ - dot_accuracy@1
72
+ - dot_accuracy@3
73
+ - dot_accuracy@5
74
+ - dot_accuracy@10
75
+ - dot_precision@1
76
+ - dot_precision@3
77
+ - dot_precision@5
78
+ - dot_precision@10
79
+ - dot_recall@1
80
+ - dot_recall@3
81
+ - dot_recall@5
82
+ - dot_recall@10
83
+ - dot_ndcg@10
84
+ - dot_mrr@10
85
+ - dot_map@100
86
+ - query_active_dims
87
+ - query_sparsity_ratio
88
+ - corpus_active_dims
89
+ - corpus_sparsity_ratio
90
+ co2_eq_emissions:
91
+ emissions: 1.4164940270091377
92
+ energy_consumed: 0.02527693261851813
93
+ source: codecarbon
94
+ training_type: fine-tuning
95
+ on_cloud: false
96
+ cpu_model: AMD Ryzen 9 6900HX with Radeon Graphics
97
+ ram_total_size: 30.6114501953125
98
+ hours_used: 0.222
99
+ hardware_used: 1 x NVIDIA GeForce RTX 3070 Ti Laptop GPU
100
+ model-index:
101
+ - name: splade-distilbert-base-uncased trained on Quora Duplicates Questions
102
+ results:
103
+ - task:
104
+ type: sparse-binary-classification
105
+ name: Sparse Binary Classification
106
+ dataset:
107
+ name: quora duplicates dev
108
+ type: quora_duplicates_dev
109
+ metrics:
110
+ - type: cosine_accuracy
111
+ value: 0.758
112
+ name: Cosine Accuracy
113
+ - type: cosine_accuracy_threshold
114
+ value: 0.8166326284408569
115
+ name: Cosine Accuracy Threshold
116
+ - type: cosine_f1
117
+ value: 0.6792899408284023
118
+ name: Cosine F1
119
+ - type: cosine_f1_threshold
120
+ value: 0.5695896148681641
121
+ name: Cosine F1 Threshold
122
+ - type: cosine_precision
123
+ value: 0.5487571701720841
124
+ name: Cosine Precision
125
+ - type: cosine_recall
126
+ value: 0.8913043478260869
127
+ name: Cosine Recall
128
+ - type: cosine_ap
129
+ value: 0.6887627674706448
130
+ name: Cosine Ap
131
+ - type: cosine_mcc
132
+ value: 0.508171027288805
133
+ name: Cosine Mcc
134
+ - type: dot_accuracy
135
+ value: 0.765
136
+ name: Dot Accuracy
137
+ - type: dot_accuracy_threshold
138
+ value: 51.6699104309082
139
+ name: Dot Accuracy Threshold
140
+ - type: dot_f1
141
+ value: 0.6762028608582575
142
+ name: Dot F1
143
+ - type: dot_f1_threshold
144
+ value: 46.524925231933594
145
+ name: Dot F1 Threshold
146
+ - type: dot_precision
147
+ value: 0.5816554809843401
148
+ name: Dot Precision
149
+ - type: dot_recall
150
+ value: 0.8074534161490683
151
+ name: Dot Recall
152
+ - type: dot_ap
153
+ value: 0.6335823489360819
154
+ name: Dot Ap
155
+ - type: dot_mcc
156
+ value: 0.4996270089694481
157
+ name: Dot Mcc
158
+ - type: euclidean_accuracy
159
+ value: 0.677
160
+ name: Euclidean Accuracy
161
+ - type: euclidean_accuracy_threshold
162
+ value: -14.272356986999512
163
+ name: Euclidean Accuracy Threshold
164
+ - type: euclidean_f1
165
+ value: 0.48599545798637395
166
+ name: Euclidean F1
167
+ - type: euclidean_f1_threshold
168
+ value: -0.6444530487060547
169
+ name: Euclidean F1 Threshold
170
+ - type: euclidean_precision
171
+ value: 0.3213213213213213
172
+ name: Euclidean Precision
173
+ - type: euclidean_recall
174
+ value: 0.9968944099378882
175
+ name: Euclidean Recall
176
+ - type: euclidean_ap
177
+ value: 0.2032823056922341
178
+ name: Euclidean Ap
179
+ - type: euclidean_mcc
180
+ value: -0.04590966956831287
181
+ name: Euclidean Mcc
182
+ - type: manhattan_accuracy
183
+ value: 0.677
184
+ name: Manhattan Accuracy
185
+ - type: manhattan_accuracy_threshold
186
+ value: -161.77682495117188
187
+ name: Manhattan Accuracy Threshold
188
+ - type: manhattan_f1
189
+ value: 0.48599545798637395
190
+ name: Manhattan F1
191
+ - type: manhattan_f1_threshold
192
+ value: -3.0494537353515625
193
+ name: Manhattan F1 Threshold
194
+ - type: manhattan_precision
195
+ value: 0.3213213213213213
196
+ name: Manhattan Precision
197
+ - type: manhattan_recall
198
+ value: 0.9968944099378882
199
+ name: Manhattan Recall
200
+ - type: manhattan_ap
201
+ value: 0.20444314945561334
202
+ name: Manhattan Ap
203
+ - type: manhattan_mcc
204
+ value: -0.04590966956831287
205
+ name: Manhattan Mcc
206
+ - type: max_accuracy
207
+ value: 0.765
208
+ name: Max Accuracy
209
+ - type: max_accuracy_threshold
210
+ value: 51.6699104309082
211
+ name: Max Accuracy Threshold
212
+ - type: max_f1
213
+ value: 0.6792899408284023
214
+ name: Max F1
215
+ - type: max_f1_threshold
216
+ value: 46.524925231933594
217
+ name: Max F1 Threshold
218
+ - type: max_precision
219
+ value: 0.5816554809843401
220
+ name: Max Precision
221
+ - type: max_recall
222
+ value: 0.9968944099378882
223
+ name: Max Recall
224
+ - type: max_ap
225
+ value: 0.6887627674706448
226
+ name: Max Ap
227
+ - type: max_mcc
228
+ value: 0.508171027288805
229
+ name: Max Mcc
230
+ - type: active_dims
231
+ value: 78.32280731201172
232
+ name: Active Dims
233
+ - type: sparsity_ratio
234
+ value: 0.9974338900690646
235
+ name: Sparsity Ratio
236
+ - task:
237
+ type: sparse-information-retrieval
238
+ name: Sparse Information Retrieval
239
+ dataset:
240
+ name: NanoMSMARCO
241
+ type: NanoMSMARCO
242
+ metrics:
243
+ - type: dot_accuracy@1
244
+ value: 0.22
245
+ name: Dot Accuracy@1
246
+ - type: dot_accuracy@3
247
+ value: 0.42
248
+ name: Dot Accuracy@3
249
+ - type: dot_accuracy@5
250
+ value: 0.52
251
+ name: Dot Accuracy@5
252
+ - type: dot_accuracy@10
253
+ value: 0.76
254
+ name: Dot Accuracy@10
255
+ - type: dot_precision@1
256
+ value: 0.22
257
+ name: Dot Precision@1
258
+ - type: dot_precision@3
259
+ value: 0.13999999999999999
260
+ name: Dot Precision@3
261
+ - type: dot_precision@5
262
+ value: 0.10400000000000001
263
+ name: Dot Precision@5
264
+ - type: dot_precision@10
265
+ value: 0.07600000000000001
266
+ name: Dot Precision@10
267
+ - type: dot_recall@1
268
+ value: 0.22
269
+ name: Dot Recall@1
270
+ - type: dot_recall@3
271
+ value: 0.42
272
+ name: Dot Recall@3
273
+ - type: dot_recall@5
274
+ value: 0.52
275
+ name: Dot Recall@5
276
+ - type: dot_recall@10
277
+ value: 0.76
278
+ name: Dot Recall@10
279
+ - type: dot_ndcg@10
280
+ value: 0.45321847177875746
281
+ name: Dot Ndcg@10
282
+ - type: dot_mrr@10
283
+ value: 0.3601269841269841
284
+ name: Dot Mrr@10
285
+ - type: dot_map@100
286
+ value: 0.37334906504034243
287
+ name: Dot Map@100
288
+ - type: query_active_dims
289
+ value: 74.76000213623047
290
+ name: Query Active Dims
291
+ - type: query_sparsity_ratio
292
+ value: 0.9975506191554868
293
+ name: Query Sparsity Ratio
294
+ - type: corpus_active_dims
295
+ value: 103.06523895263672
296
+ name: Corpus Active Dims
297
+ - type: corpus_sparsity_ratio
298
+ value: 0.9966232475279261
299
+ name: Corpus Sparsity Ratio
300
+ - type: dot_accuracy@1
301
+ value: 0.22
302
+ name: Dot Accuracy@1
303
+ - type: dot_accuracy@3
304
+ value: 0.42
305
+ name: Dot Accuracy@3
306
+ - type: dot_accuracy@5
307
+ value: 0.52
308
+ name: Dot Accuracy@5
309
+ - type: dot_accuracy@10
310
+ value: 0.76
311
+ name: Dot Accuracy@10
312
+ - type: dot_precision@1
313
+ value: 0.22
314
+ name: Dot Precision@1
315
+ - type: dot_precision@3
316
+ value: 0.13999999999999999
317
+ name: Dot Precision@3
318
+ - type: dot_precision@5
319
+ value: 0.10400000000000001
320
+ name: Dot Precision@5
321
+ - type: dot_precision@10
322
+ value: 0.07600000000000001
323
+ name: Dot Precision@10
324
+ - type: dot_recall@1
325
+ value: 0.22
326
+ name: Dot Recall@1
327
+ - type: dot_recall@3
328
+ value: 0.42
329
+ name: Dot Recall@3
330
+ - type: dot_recall@5
331
+ value: 0.52
332
+ name: Dot Recall@5
333
+ - type: dot_recall@10
334
+ value: 0.76
335
+ name: Dot Recall@10
336
+ - type: dot_ndcg@10
337
+ value: 0.45321847177875746
338
+ name: Dot Ndcg@10
339
+ - type: dot_mrr@10
340
+ value: 0.3601269841269841
341
+ name: Dot Mrr@10
342
+ - type: dot_map@100
343
+ value: 0.37334906504034243
344
+ name: Dot Map@100
345
+ - type: query_active_dims
346
+ value: 74.76000213623047
347
+ name: Query Active Dims
348
+ - type: query_sparsity_ratio
349
+ value: 0.9975506191554868
350
+ name: Query Sparsity Ratio
351
+ - type: corpus_active_dims
352
+ value: 103.06523895263672
353
+ name: Corpus Active Dims
354
+ - type: corpus_sparsity_ratio
355
+ value: 0.9966232475279261
356
+ name: Corpus Sparsity Ratio
357
+ - task:
358
+ type: sparse-information-retrieval
359
+ name: Sparse Information Retrieval
360
+ dataset:
361
+ name: NanoNQ
362
+ type: NanoNQ
363
+ metrics:
364
+ - type: dot_accuracy@1
365
+ value: 0.38
366
+ name: Dot Accuracy@1
367
+ - type: dot_accuracy@3
368
+ value: 0.54
369
+ name: Dot Accuracy@3
370
+ - type: dot_accuracy@5
371
+ value: 0.62
372
+ name: Dot Accuracy@5
373
+ - type: dot_accuracy@10
374
+ value: 0.62
375
+ name: Dot Accuracy@10
376
+ - type: dot_precision@1
377
+ value: 0.38
378
+ name: Dot Precision@1
379
+ - type: dot_precision@3
380
+ value: 0.18
381
+ name: Dot Precision@3
382
+ - type: dot_precision@5
383
+ value: 0.12400000000000003
384
+ name: Dot Precision@5
385
+ - type: dot_precision@10
386
+ value: 0.06400000000000002
387
+ name: Dot Precision@10
388
+ - type: dot_recall@1
389
+ value: 0.36
390
+ name: Dot Recall@1
391
+ - type: dot_recall@3
392
+ value: 0.52
393
+ name: Dot Recall@3
394
+ - type: dot_recall@5
395
+ value: 0.6
396
+ name: Dot Recall@5
397
+ - type: dot_recall@10
398
+ value: 0.61
399
+ name: Dot Recall@10
400
+ - type: dot_ndcg@10
401
+ value: 0.4828377104499333
402
+ name: Dot Ndcg@10
403
+ - type: dot_mrr@10
404
+ value: 0.4536666666666666
405
+ name: Dot Mrr@10
406
+ - type: dot_map@100
407
+ value: 0.445384784044708
408
+ name: Dot Map@100
409
+ - type: query_active_dims
410
+ value: 74.73999786376953
411
+ name: Query Active Dims
412
+ - type: query_sparsity_ratio
413
+ value: 0.9975512745605213
414
+ name: Query Sparsity Ratio
415
+ - type: corpus_active_dims
416
+ value: 141.31478881835938
417
+ name: Corpus Active Dims
418
+ - type: corpus_sparsity_ratio
419
+ value: 0.9953700678586476
420
+ name: Corpus Sparsity Ratio
421
+ - type: dot_accuracy@1
422
+ value: 0.38
423
+ name: Dot Accuracy@1
424
+ - type: dot_accuracy@3
425
+ value: 0.54
426
+ name: Dot Accuracy@3
427
+ - type: dot_accuracy@5
428
+ value: 0.62
429
+ name: Dot Accuracy@5
430
+ - type: dot_accuracy@10
431
+ value: 0.62
432
+ name: Dot Accuracy@10
433
+ - type: dot_precision@1
434
+ value: 0.38
435
+ name: Dot Precision@1
436
+ - type: dot_precision@3
437
+ value: 0.18
438
+ name: Dot Precision@3
439
+ - type: dot_precision@5
440
+ value: 0.12400000000000003
441
+ name: Dot Precision@5
442
+ - type: dot_precision@10
443
+ value: 0.06400000000000002
444
+ name: Dot Precision@10
445
+ - type: dot_recall@1
446
+ value: 0.36
447
+ name: Dot Recall@1
448
+ - type: dot_recall@3
449
+ value: 0.52
450
+ name: Dot Recall@3
451
+ - type: dot_recall@5
452
+ value: 0.6
453
+ name: Dot Recall@5
454
+ - type: dot_recall@10
455
+ value: 0.61
456
+ name: Dot Recall@10
457
+ - type: dot_ndcg@10
458
+ value: 0.4828377104499333
459
+ name: Dot Ndcg@10
460
+ - type: dot_mrr@10
461
+ value: 0.4536666666666666
462
+ name: Dot Mrr@10
463
+ - type: dot_map@100
464
+ value: 0.445384784044708
465
+ name: Dot Map@100
466
+ - type: query_active_dims
467
+ value: 74.73999786376953
468
+ name: Query Active Dims
469
+ - type: query_sparsity_ratio
470
+ value: 0.9975512745605213
471
+ name: Query Sparsity Ratio
472
+ - type: corpus_active_dims
473
+ value: 141.31478881835938
474
+ name: Corpus Active Dims
475
+ - type: corpus_sparsity_ratio
476
+ value: 0.9953700678586476
477
+ name: Corpus Sparsity Ratio
478
+ - task:
479
+ type: sparse-information-retrieval
480
+ name: Sparse Information Retrieval
481
+ dataset:
482
+ name: NanoNFCorpus
483
+ type: NanoNFCorpus
484
+ metrics:
485
+ - type: dot_accuracy@1
486
+ value: 0.34
487
+ name: Dot Accuracy@1
488
+ - type: dot_accuracy@3
489
+ value: 0.5
490
+ name: Dot Accuracy@3
491
+ - type: dot_accuracy@5
492
+ value: 0.54
493
+ name: Dot Accuracy@5
494
+ - type: dot_accuracy@10
495
+ value: 0.58
496
+ name: Dot Accuracy@10
497
+ - type: dot_precision@1
498
+ value: 0.34
499
+ name: Dot Precision@1
500
+ - type: dot_precision@3
501
+ value: 0.30666666666666664
502
+ name: Dot Precision@3
503
+ - type: dot_precision@5
504
+ value: 0.26
505
+ name: Dot Precision@5
506
+ - type: dot_precision@10
507
+ value: 0.198
508
+ name: Dot Precision@10
509
+ - type: dot_recall@1
510
+ value: 0.011597172822497613
511
+ name: Dot Recall@1
512
+ - type: dot_recall@3
513
+ value: 0.06058581579610722
514
+ name: Dot Recall@3
515
+ - type: dot_recall@5
516
+ value: 0.08260772201759854
517
+ name: Dot Recall@5
518
+ - type: dot_recall@10
519
+ value: 0.09800124609193644
520
+ name: Dot Recall@10
521
+ - type: dot_ndcg@10
522
+ value: 0.2466972614666078
523
+ name: Dot Ndcg@10
524
+ - type: dot_mrr@10
525
+ value: 0.42200000000000004
526
+ name: Dot Mrr@10
527
+ - type: dot_map@100
528
+ value: 0.09401937795309984
529
+ name: Dot Map@100
530
+ - type: query_active_dims
531
+ value: 79.69999694824219
532
+ name: Query Active Dims
533
+ - type: query_sparsity_ratio
534
+ value: 0.9973887688569477
535
+ name: Query Sparsity Ratio
536
+ - type: corpus_active_dims
537
+ value: 202.17269897460938
538
+ name: Corpus Active Dims
539
+ - type: corpus_sparsity_ratio
540
+ value: 0.9933761647672298
541
+ name: Corpus Sparsity Ratio
542
+ - type: dot_accuracy@1
543
+ value: 0.34
544
+ name: Dot Accuracy@1
545
+ - type: dot_accuracy@3
546
+ value: 0.5
547
+ name: Dot Accuracy@3
548
+ - type: dot_accuracy@5
549
+ value: 0.54
550
+ name: Dot Accuracy@5
551
+ - type: dot_accuracy@10
552
+ value: 0.58
553
+ name: Dot Accuracy@10
554
+ - type: dot_precision@1
555
+ value: 0.34
556
+ name: Dot Precision@1
557
+ - type: dot_precision@3
558
+ value: 0.30666666666666664
559
+ name: Dot Precision@3
560
+ - type: dot_precision@5
561
+ value: 0.26
562
+ name: Dot Precision@5
563
+ - type: dot_precision@10
564
+ value: 0.198
565
+ name: Dot Precision@10
566
+ - type: dot_recall@1
567
+ value: 0.011597172822497613
568
+ name: Dot Recall@1
569
+ - type: dot_recall@3
570
+ value: 0.06058581579610722
571
+ name: Dot Recall@3
572
+ - type: dot_recall@5
573
+ value: 0.08260772201759854
574
+ name: Dot Recall@5
575
+ - type: dot_recall@10
576
+ value: 0.09800124609193644
577
+ name: Dot Recall@10
578
+ - type: dot_ndcg@10
579
+ value: 0.2466972614666078
580
+ name: Dot Ndcg@10
581
+ - type: dot_mrr@10
582
+ value: 0.42200000000000004
583
+ name: Dot Mrr@10
584
+ - type: dot_map@100
585
+ value: 0.09401937795309984
586
+ name: Dot Map@100
587
+ - type: query_active_dims
588
+ value: 79.69999694824219
589
+ name: Query Active Dims
590
+ - type: query_sparsity_ratio
591
+ value: 0.9973887688569477
592
+ name: Query Sparsity Ratio
593
+ - type: corpus_active_dims
594
+ value: 202.17269897460938
595
+ name: Corpus Active Dims
596
+ - type: corpus_sparsity_ratio
597
+ value: 0.9933761647672298
598
+ name: Corpus Sparsity Ratio
599
+ - task:
600
+ type: sparse-information-retrieval
601
+ name: Sparse Information Retrieval
602
+ dataset:
603
+ name: NanoQuoraRetrieval
604
+ type: NanoQuoraRetrieval
605
+ metrics:
606
+ - type: dot_accuracy@1
607
+ value: 0.94
608
+ name: Dot Accuracy@1
609
+ - type: dot_accuracy@3
610
+ value: 0.98
611
+ name: Dot Accuracy@3
612
+ - type: dot_accuracy@5
613
+ value: 0.98
614
+ name: Dot Accuracy@5
615
+ - type: dot_accuracy@10
616
+ value: 0.98
617
+ name: Dot Accuracy@10
618
+ - type: dot_precision@1
619
+ value: 0.94
620
+ name: Dot Precision@1
621
+ - type: dot_precision@3
622
+ value: 0.3933333333333333
623
+ name: Dot Precision@3
624
+ - type: dot_precision@5
625
+ value: 0.24799999999999997
626
+ name: Dot Precision@5
627
+ - type: dot_precision@10
628
+ value: 0.13199999999999998
629
+ name: Dot Precision@10
630
+ - type: dot_recall@1
631
+ value: 0.8173333333333332
632
+ name: Dot Recall@1
633
+ - type: dot_recall@3
634
+ value: 0.9279999999999999
635
+ name: Dot Recall@3
636
+ - type: dot_recall@5
637
+ value: 0.946
638
+ name: Dot Recall@5
639
+ - type: dot_recall@10
640
+ value: 0.97
641
+ name: Dot Recall@10
642
+ - type: dot_ndcg@10
643
+ value: 0.9467235239993945
644
+ name: Dot Ndcg@10
645
+ - type: dot_mrr@10
646
+ value: 0.96
647
+ name: Dot Mrr@10
648
+ - type: dot_map@100
649
+ value: 0.9290737327188939
650
+ name: Dot Map@100
651
+ - type: query_active_dims
652
+ value: 76.58000183105469
653
+ name: Query Active Dims
654
+ - type: query_sparsity_ratio
655
+ value: 0.9974909900455063
656
+ name: Query Sparsity Ratio
657
+ - type: corpus_active_dims
658
+ value: 77.59056854248047
659
+ name: Corpus Active Dims
660
+ - type: corpus_sparsity_ratio
661
+ value: 0.9974578805929336
662
+ name: Corpus Sparsity Ratio
663
+ - type: dot_accuracy@1
664
+ value: 0.94
665
+ name: Dot Accuracy@1
666
+ - type: dot_accuracy@3
667
+ value: 0.98
668
+ name: Dot Accuracy@3
669
+ - type: dot_accuracy@5
670
+ value: 0.98
671
+ name: Dot Accuracy@5
672
+ - type: dot_accuracy@10
673
+ value: 0.98
674
+ name: Dot Accuracy@10
675
+ - type: dot_precision@1
676
+ value: 0.94
677
+ name: Dot Precision@1
678
+ - type: dot_precision@3
679
+ value: 0.3933333333333333
680
+ name: Dot Precision@3
681
+ - type: dot_precision@5
682
+ value: 0.24799999999999997
683
+ name: Dot Precision@5
684
+ - type: dot_precision@10
685
+ value: 0.13199999999999998
686
+ name: Dot Precision@10
687
+ - type: dot_recall@1
688
+ value: 0.8173333333333332
689
+ name: Dot Recall@1
690
+ - type: dot_recall@3
691
+ value: 0.9279999999999999
692
+ name: Dot Recall@3
693
+ - type: dot_recall@5
694
+ value: 0.946
695
+ name: Dot Recall@5
696
+ - type: dot_recall@10
697
+ value: 0.97
698
+ name: Dot Recall@10
699
+ - type: dot_ndcg@10
700
+ value: 0.9467235239993945
701
+ name: Dot Ndcg@10
702
+ - type: dot_mrr@10
703
+ value: 0.96
704
+ name: Dot Mrr@10
705
+ - type: dot_map@100
706
+ value: 0.9290737327188939
707
+ name: Dot Map@100
708
+ - type: query_active_dims
709
+ value: 76.58000183105469
710
+ name: Query Active Dims
711
+ - type: query_sparsity_ratio
712
+ value: 0.9974909900455063
713
+ name: Query Sparsity Ratio
714
+ - type: corpus_active_dims
715
+ value: 77.59056854248047
716
+ name: Corpus Active Dims
717
+ - type: corpus_sparsity_ratio
718
+ value: 0.9974578805929336
719
+ name: Corpus Sparsity Ratio
720
+ - task:
721
+ type: sparse-nano-beir
722
+ name: Sparse Nano BEIR
723
+ dataset:
724
+ name: NanoBEIR mean
725
+ type: NanoBEIR_mean
726
+ metrics:
727
+ - type: dot_accuracy@1
728
+ value: 0.47
729
+ name: Dot Accuracy@1
730
+ - type: dot_accuracy@3
731
+ value: 0.61
732
+ name: Dot Accuracy@3
733
+ - type: dot_accuracy@5
734
+ value: 0.665
735
+ name: Dot Accuracy@5
736
+ - type: dot_accuracy@10
737
+ value: 0.735
738
+ name: Dot Accuracy@10
739
+ - type: dot_precision@1
740
+ value: 0.47
741
+ name: Dot Precision@1
742
+ - type: dot_precision@3
743
+ value: 0.255
744
+ name: Dot Precision@3
745
+ - type: dot_precision@5
746
+ value: 0.184
747
+ name: Dot Precision@5
748
+ - type: dot_precision@10
749
+ value: 0.1175
750
+ name: Dot Precision@10
751
+ - type: dot_recall@1
752
+ value: 0.3522326265389577
753
+ name: Dot Recall@1
754
+ - type: dot_recall@3
755
+ value: 0.4821464539490268
756
+ name: Dot Recall@3
757
+ - type: dot_recall@5
758
+ value: 0.5371519305043997
759
+ name: Dot Recall@5
760
+ - type: dot_recall@10
761
+ value: 0.6095003115229841
762
+ name: Dot Recall@10
763
+ - type: dot_ndcg@10
764
+ value: 0.5323692419236733
765
+ name: Dot Ndcg@10
766
+ - type: dot_mrr@10
767
+ value: 0.5489484126984127
768
+ name: Dot Mrr@10
769
+ - type: dot_map@100
770
+ value: 0.46045673993926106
771
+ name: Dot Map@100
772
+ - type: query_active_dims
773
+ value: 76.44499969482422
774
+ name: Query Active Dims
775
+ - type: query_sparsity_ratio
776
+ value: 0.9974954131546155
777
+ name: Query Sparsity Ratio
778
+ - type: corpus_active_dims
779
+ value: 122.79780664247188
780
+ name: Corpus Active Dims
781
+ - type: corpus_sparsity_ratio
782
+ value: 0.9959767444255792
783
+ name: Corpus Sparsity Ratio
784
+ - type: dot_accuracy@1
785
+ value: 0.4359811616954475
786
+ name: Dot Accuracy@1
787
+ - type: dot_accuracy@3
788
+ value: 0.6088540031397174
789
+ name: Dot Accuracy@3
790
+ - type: dot_accuracy@5
791
+ value: 0.6659026687598116
792
+ name: Dot Accuracy@5
793
+ - type: dot_accuracy@10
794
+ value: 0.7383987441130299
795
+ name: Dot Accuracy@10
796
+ - type: dot_precision@1
797
+ value: 0.4359811616954475
798
+ name: Dot Precision@1
799
+ - type: dot_precision@3
800
+ value: 0.2725170068027211
801
+ name: Dot Precision@3
802
+ - type: dot_precision@5
803
+ value: 0.2089481946624804
804
+ name: Dot Precision@5
805
+ - type: dot_precision@10
806
+ value: 0.14605965463108322
807
+ name: Dot Precision@10
808
+ - type: dot_recall@1
809
+ value: 0.2532746332292894
810
+ name: Dot Recall@1
811
+ - type: dot_recall@3
812
+ value: 0.3813452238818861
813
+ name: Dot Recall@3
814
+ - type: dot_recall@5
815
+ value: 0.4363867898661836
816
+ name: Dot Recall@5
817
+ - type: dot_recall@10
818
+ value: 0.5099503000039356
819
+ name: Dot Recall@10
820
+ - type: dot_ndcg@10
821
+ value: 0.4684519639817077
822
+ name: Dot Ndcg@10
823
+ - type: dot_mrr@10
824
+ value: 0.5328029827315542
825
+ name: Dot Mrr@10
826
+ - type: dot_map@100
827
+ value: 0.39738635557561647
828
+ name: Dot Map@100
829
+ - type: query_active_dims
830
+ value: 90.39137197532713
831
+ name: Query Active Dims
832
+ - type: query_sparsity_ratio
833
+ value: 0.9970384846348428
834
+ name: Query Sparsity Ratio
835
+ - type: corpus_active_dims
836
+ value: 152.36685474307478
837
+ name: Corpus Active Dims
838
+ - type: corpus_sparsity_ratio
839
+ value: 0.9950079662295042
840
+ name: Corpus Sparsity Ratio
841
+ - task:
842
+ type: sparse-information-retrieval
843
+ name: Sparse Information Retrieval
844
+ dataset:
845
+ name: NanoClimateFEVER
846
+ type: NanoClimateFEVER
847
+ metrics:
848
+ - type: dot_accuracy@1
849
+ value: 0.18
850
+ name: Dot Accuracy@1
851
+ - type: dot_accuracy@3
852
+ value: 0.32
853
+ name: Dot Accuracy@3
854
+ - type: dot_accuracy@5
855
+ value: 0.4
856
+ name: Dot Accuracy@5
857
+ - type: dot_accuracy@10
858
+ value: 0.48
859
+ name: Dot Accuracy@10
860
+ - type: dot_precision@1
861
+ value: 0.18
862
+ name: Dot Precision@1
863
+ - type: dot_precision@3
864
+ value: 0.10666666666666666
865
+ name: Dot Precision@3
866
+ - type: dot_precision@5
867
+ value: 0.08400000000000002
868
+ name: Dot Precision@5
869
+ - type: dot_precision@10
870
+ value: 0.054000000000000006
871
+ name: Dot Precision@10
872
+ - type: dot_recall@1
873
+ value: 0.085
874
+ name: Dot Recall@1
875
+ - type: dot_recall@3
876
+ value: 0.14666666666666667
877
+ name: Dot Recall@3
878
+ - type: dot_recall@5
879
+ value: 0.17833333333333332
880
+ name: Dot Recall@5
881
+ - type: dot_recall@10
882
+ value: 0.215
883
+ name: Dot Recall@10
884
+ - type: dot_ndcg@10
885
+ value: 0.1845115403570178
886
+ name: Dot Ndcg@10
887
+ - type: dot_mrr@10
888
+ value: 0.2674126984126984
889
+ name: Dot Mrr@10
890
+ - type: dot_map@100
891
+ value: 0.1475834110231865
892
+ name: Dot Map@100
893
+ - type: query_active_dims
894
+ value: 89.86000061035156
895
+ name: Query Active Dims
896
+ - type: query_sparsity_ratio
897
+ value: 0.9970558940891701
898
+ name: Query Sparsity Ratio
899
+ - type: corpus_active_dims
900
+ value: 221.75527954101562
901
+ name: Corpus Active Dims
902
+ - type: corpus_sparsity_ratio
903
+ value: 0.992734575730915
904
+ name: Corpus Sparsity Ratio
905
+ - task:
906
+ type: sparse-information-retrieval
907
+ name: Sparse Information Retrieval
908
+ dataset:
909
+ name: NanoDBPedia
910
+ type: NanoDBPedia
911
+ metrics:
912
+ - type: dot_accuracy@1
913
+ value: 0.6
914
+ name: Dot Accuracy@1
915
+ - type: dot_accuracy@3
916
+ value: 0.84
917
+ name: Dot Accuracy@3
918
+ - type: dot_accuracy@5
919
+ value: 0.84
920
+ name: Dot Accuracy@5
921
+ - type: dot_accuracy@10
922
+ value: 0.92
923
+ name: Dot Accuracy@10
924
+ - type: dot_precision@1
925
+ value: 0.6
926
+ name: Dot Precision@1
927
+ - type: dot_precision@3
928
+ value: 0.5266666666666666
929
+ name: Dot Precision@3
930
+ - type: dot_precision@5
931
+ value: 0.456
932
+ name: Dot Precision@5
933
+ - type: dot_precision@10
934
+ value: 0.4220000000000001
935
+ name: Dot Precision@10
936
+ - type: dot_recall@1
937
+ value: 0.04570544957623723
938
+ name: Dot Recall@1
939
+ - type: dot_recall@3
940
+ value: 0.15367137863132574
941
+ name: Dot Recall@3
942
+ - type: dot_recall@5
943
+ value: 0.1908008582920462
944
+ name: Dot Recall@5
945
+ - type: dot_recall@10
946
+ value: 0.293554014064817
947
+ name: Dot Recall@10
948
+ - type: dot_ndcg@10
949
+ value: 0.5070720730882787
950
+ name: Dot Ndcg@10
951
+ - type: dot_mrr@10
952
+ value: 0.7147222222222225
953
+ name: Dot Mrr@10
954
+ - type: dot_map@100
955
+ value: 0.3906658166774757
956
+ name: Dot Map@100
957
+ - type: query_active_dims
958
+ value: 69.5199966430664
959
+ name: Query Active Dims
960
+ - type: query_sparsity_ratio
961
+ value: 0.997722298779796
962
+ name: Query Sparsity Ratio
963
+ - type: corpus_active_dims
964
+ value: 135.93350219726562
965
+ name: Corpus Active Dims
966
+ - type: corpus_sparsity_ratio
967
+ value: 0.9955463763122578
968
+ name: Corpus Sparsity Ratio
969
+ - task:
970
+ type: sparse-information-retrieval
971
+ name: Sparse Information Retrieval
972
+ dataset:
973
+ name: NanoFEVER
974
+ type: NanoFEVER
975
+ metrics:
976
+ - type: dot_accuracy@1
977
+ value: 0.58
978
+ name: Dot Accuracy@1
979
+ - type: dot_accuracy@3
980
+ value: 0.76
981
+ name: Dot Accuracy@3
982
+ - type: dot_accuracy@5
983
+ value: 0.8
984
+ name: Dot Accuracy@5
985
+ - type: dot_accuracy@10
986
+ value: 0.86
987
+ name: Dot Accuracy@10
988
+ - type: dot_precision@1
989
+ value: 0.58
990
+ name: Dot Precision@1
991
+ - type: dot_precision@3
992
+ value: 0.26666666666666666
993
+ name: Dot Precision@3
994
+ - type: dot_precision@5
995
+ value: 0.16799999999999998
996
+ name: Dot Precision@5
997
+ - type: dot_precision@10
998
+ value: 0.09
999
+ name: Dot Precision@10
1000
+ - type: dot_recall@1
1001
+ value: 0.5466666666666666
1002
+ name: Dot Recall@1
1003
+ - type: dot_recall@3
1004
+ value: 0.7466666666666667
1005
+ name: Dot Recall@3
1006
+ - type: dot_recall@5
1007
+ value: 0.7866666666666667
1008
+ name: Dot Recall@5
1009
+ - type: dot_recall@10
1010
+ value: 0.8466666666666667
1011
+ name: Dot Recall@10
1012
+ - type: dot_ndcg@10
1013
+ value: 0.7069849294263234
1014
+ name: Dot Ndcg@10
1015
+ - type: dot_mrr@10
1016
+ value: 0.6765000000000001
1017
+ name: Dot Mrr@10
1018
+ - type: dot_map@100
1019
+ value: 0.6651380090497737
1020
+ name: Dot Map@100
1021
+ - type: query_active_dims
1022
+ value: 89.87999725341797
1023
+ name: Query Active Dims
1024
+ - type: query_sparsity_ratio
1025
+ value: 0.9970552389340994
1026
+ name: Query Sparsity Ratio
1027
+ - type: corpus_active_dims
1028
+ value: 221.215576171875
1029
+ name: Corpus Active Dims
1030
+ - type: corpus_sparsity_ratio
1031
+ value: 0.9927522581688004
1032
+ name: Corpus Sparsity Ratio
1033
+ - task:
1034
+ type: sparse-information-retrieval
1035
+ name: Sparse Information Retrieval
1036
+ dataset:
1037
+ name: NanoFiQA2018
1038
+ type: NanoFiQA2018
1039
+ metrics:
1040
+ - type: dot_accuracy@1
1041
+ value: 0.28
1042
+ name: Dot Accuracy@1
1043
+ - type: dot_accuracy@3
1044
+ value: 0.42
1045
+ name: Dot Accuracy@3
1046
+ - type: dot_accuracy@5
1047
+ value: 0.46
1048
+ name: Dot Accuracy@5
1049
+ - type: dot_accuracy@10
1050
+ value: 0.5
1051
+ name: Dot Accuracy@10
1052
+ - type: dot_precision@1
1053
+ value: 0.28
1054
+ name: Dot Precision@1
1055
+ - type: dot_precision@3
1056
+ value: 0.18
1057
+ name: Dot Precision@3
1058
+ - type: dot_precision@5
1059
+ value: 0.136
1060
+ name: Dot Precision@5
1061
+ - type: dot_precision@10
1062
+ value: 0.08399999999999999
1063
+ name: Dot Precision@10
1064
+ - type: dot_recall@1
1065
+ value: 0.14183333333333334
1066
+ name: Dot Recall@1
1067
+ - type: dot_recall@3
1068
+ value: 0.24288888888888888
1069
+ name: Dot Recall@3
1070
+ - type: dot_recall@5
1071
+ value: 0.27715873015873016
1072
+ name: Dot Recall@5
1073
+ - type: dot_recall@10
1074
+ value: 0.3288730158730159
1075
+ name: Dot Recall@10
1076
+ - type: dot_ndcg@10
1077
+ value: 0.28813286680239514
1078
+ name: Dot Ndcg@10
1079
+ - type: dot_mrr@10
1080
+ value: 0.3561904761904763
1081
+ name: Dot Mrr@10
1082
+ - type: dot_map@100
1083
+ value: 0.2415362537997973
1084
+ name: Dot Map@100
1085
+ - type: query_active_dims
1086
+ value: 82.86000061035156
1087
+ name: Query Active Dims
1088
+ - type: query_sparsity_ratio
1089
+ value: 0.9972852368583202
1090
+ name: Query Sparsity Ratio
1091
+ - type: corpus_active_dims
1092
+ value: 130.93699645996094
1093
+ name: Corpus Active Dims
1094
+ - type: corpus_sparsity_ratio
1095
+ value: 0.9957100780925245
1096
+ name: Corpus Sparsity Ratio
1097
+ - task:
1098
+ type: sparse-information-retrieval
1099
+ name: Sparse Information Retrieval
1100
+ dataset:
1101
+ name: NanoHotpotQA
1102
+ type: NanoHotpotQA
1103
+ metrics:
1104
+ - type: dot_accuracy@1
1105
+ value: 0.78
1106
+ name: Dot Accuracy@1
1107
+ - type: dot_accuracy@3
1108
+ value: 0.84
1109
+ name: Dot Accuracy@3
1110
+ - type: dot_accuracy@5
1111
+ value: 0.92
1112
+ name: Dot Accuracy@5
1113
+ - type: dot_accuracy@10
1114
+ value: 0.98
1115
+ name: Dot Accuracy@10
1116
+ - type: dot_precision@1
1117
+ value: 0.78
1118
+ name: Dot Precision@1
1119
+ - type: dot_precision@3
1120
+ value: 0.3733333333333333
1121
+ name: Dot Precision@3
1122
+ - type: dot_precision@5
1123
+ value: 0.28400000000000003
1124
+ name: Dot Precision@5
1125
+ - type: dot_precision@10
1126
+ value: 0.16
1127
+ name: Dot Precision@10
1128
+ - type: dot_recall@1
1129
+ value: 0.39
1130
+ name: Dot Recall@1
1131
+ - type: dot_recall@3
1132
+ value: 0.56
1133
+ name: Dot Recall@3
1134
+ - type: dot_recall@5
1135
+ value: 0.71
1136
+ name: Dot Recall@5
1137
+ - type: dot_recall@10
1138
+ value: 0.8
1139
+ name: Dot Recall@10
1140
+ - type: dot_ndcg@10
1141
+ value: 0.7143331285788386
1142
+ name: Dot Ndcg@10
1143
+ - type: dot_mrr@10
1144
+ value: 0.8361904761904762
1145
+ name: Dot Mrr@10
1146
+ - type: dot_map@100
1147
+ value: 0.6181181734895289
1148
+ name: Dot Map@100
1149
+ - type: query_active_dims
1150
+ value: 91.9800033569336
1151
+ name: Query Active Dims
1152
+ - type: query_sparsity_ratio
1153
+ value: 0.9969864359033833
1154
+ name: Query Sparsity Ratio
1155
+ - type: corpus_active_dims
1156
+ value: 152.01571655273438
1157
+ name: Corpus Active Dims
1158
+ - type: corpus_sparsity_ratio
1159
+ value: 0.9950194706587794
1160
+ name: Corpus Sparsity Ratio
1161
+ - task:
1162
+ type: sparse-information-retrieval
1163
+ name: Sparse Information Retrieval
1164
+ dataset:
1165
+ name: NanoSCIDOCS
1166
+ type: NanoSCIDOCS
1167
+ metrics:
1168
+ - type: dot_accuracy@1
1169
+ value: 0.36
1170
+ name: Dot Accuracy@1
1171
+ - type: dot_accuracy@3
1172
+ value: 0.58
1173
+ name: Dot Accuracy@3
1174
+ - type: dot_accuracy@5
1175
+ value: 0.68
1176
+ name: Dot Accuracy@5
1177
+ - type: dot_accuracy@10
1178
+ value: 0.76
1179
+ name: Dot Accuracy@10
1180
+ - type: dot_precision@1
1181
+ value: 0.36
1182
+ name: Dot Precision@1
1183
+ - type: dot_precision@3
1184
+ value: 0.2733333333333333
1185
+ name: Dot Precision@3
1186
+ - type: dot_precision@5
1187
+ value: 0.21199999999999997
1188
+ name: Dot Precision@5
1189
+ - type: dot_precision@10
1190
+ value: 0.15199999999999997
1191
+ name: Dot Precision@10
1192
+ - type: dot_recall@1
1193
+ value: 0.07566666666666666
1194
+ name: Dot Recall@1
1195
+ - type: dot_recall@3
1196
+ value: 0.16966666666666666
1197
+ name: Dot Recall@3
1198
+ - type: dot_recall@5
1199
+ value: 0.21766666666666665
1200
+ name: Dot Recall@5
1201
+ - type: dot_recall@10
1202
+ value: 0.31066666666666665
1203
+ name: Dot Recall@10
1204
+ - type: dot_ndcg@10
1205
+ value: 0.30291194083231554
1206
+ name: Dot Ndcg@10
1207
+ - type: dot_mrr@10
1208
+ value: 0.4943888888888889
1209
+ name: Dot Mrr@10
1210
+ - type: dot_map@100
1211
+ value: 0.21666464487074008
1212
+ name: Dot Map@100
1213
+ - type: query_active_dims
1214
+ value: 94.30000305175781
1215
+ name: Query Active Dims
1216
+ - type: query_sparsity_ratio
1217
+ value: 0.996910425167035
1218
+ name: Query Sparsity Ratio
1219
+ - type: corpus_active_dims
1220
+ value: 199.64630126953125
1221
+ name: Corpus Active Dims
1222
+ - type: corpus_sparsity_ratio
1223
+ value: 0.9934589377737524
1224
+ name: Corpus Sparsity Ratio
1225
+ - task:
1226
+ type: sparse-information-retrieval
1227
+ name: Sparse Information Retrieval
1228
+ dataset:
1229
+ name: NanoArguAna
1230
+ type: NanoArguAna
1231
+ metrics:
1232
+ - type: dot_accuracy@1
1233
+ value: 0.1
1234
+ name: Dot Accuracy@1
1235
+ - type: dot_accuracy@3
1236
+ value: 0.34
1237
+ name: Dot Accuracy@3
1238
+ - type: dot_accuracy@5
1239
+ value: 0.42
1240
+ name: Dot Accuracy@5
1241
+ - type: dot_accuracy@10
1242
+ value: 0.44
1243
+ name: Dot Accuracy@10
1244
+ - type: dot_precision@1
1245
+ value: 0.1
1246
+ name: Dot Precision@1
1247
+ - type: dot_precision@3
1248
+ value: 0.1133333333333333
1249
+ name: Dot Precision@3
1250
+ - type: dot_precision@5
1251
+ value: 0.084
1252
+ name: Dot Precision@5
1253
+ - type: dot_precision@10
1254
+ value: 0.044000000000000004
1255
+ name: Dot Precision@10
1256
+ - type: dot_recall@1
1257
+ value: 0.1
1258
+ name: Dot Recall@1
1259
+ - type: dot_recall@3
1260
+ value: 0.34
1261
+ name: Dot Recall@3
1262
+ - type: dot_recall@5
1263
+ value: 0.42
1264
+ name: Dot Recall@5
1265
+ - type: dot_recall@10
1266
+ value: 0.44
1267
+ name: Dot Recall@10
1268
+ - type: dot_ndcg@10
1269
+ value: 0.2781554838544819
1270
+ name: Dot Ndcg@10
1271
+ - type: dot_mrr@10
1272
+ value: 0.22466666666666665
1273
+ name: Dot Mrr@10
1274
+ - type: dot_map@100
1275
+ value: 0.2332757160696607
1276
+ name: Dot Map@100
1277
+ - type: query_active_dims
1278
+ value: 189.10000610351562
1279
+ name: Query Active Dims
1280
+ - type: query_sparsity_ratio
1281
+ value: 0.9938044687077021
1282
+ name: Query Sparsity Ratio
1283
+ - type: corpus_active_dims
1284
+ value: 164.03329467773438
1285
+ name: Corpus Active Dims
1286
+ - type: corpus_sparsity_ratio
1287
+ value: 0.9946257357093985
1288
+ name: Corpus Sparsity Ratio
1289
+ - task:
1290
+ type: sparse-information-retrieval
1291
+ name: Sparse Information Retrieval
1292
+ dataset:
1293
+ name: NanoSciFact
1294
+ type: NanoSciFact
1295
+ metrics:
1296
+ - type: dot_accuracy@1
1297
+ value: 0.52
1298
+ name: Dot Accuracy@1
1299
+ - type: dot_accuracy@3
1300
+ value: 0.62
1301
+ name: Dot Accuracy@3
1302
+ - type: dot_accuracy@5
1303
+ value: 0.64
1304
+ name: Dot Accuracy@5
1305
+ - type: dot_accuracy@10
1306
+ value: 0.76
1307
+ name: Dot Accuracy@10
1308
+ - type: dot_precision@1
1309
+ value: 0.52
1310
+ name: Dot Precision@1
1311
+ - type: dot_precision@3
1312
+ value: 0.21333333333333332
1313
+ name: Dot Precision@3
1314
+ - type: dot_precision@5
1315
+ value: 0.14
1316
+ name: Dot Precision@5
1317
+ - type: dot_precision@10
1318
+ value: 0.08399999999999999
1319
+ name: Dot Precision@10
1320
+ - type: dot_recall@1
1321
+ value: 0.475
1322
+ name: Dot Recall@1
1323
+ - type: dot_recall@3
1324
+ value: 0.58
1325
+ name: Dot Recall@3
1326
+ - type: dot_recall@5
1327
+ value: 0.615
1328
+ name: Dot Recall@5
1329
+ - type: dot_recall@10
1330
+ value: 0.74
1331
+ name: Dot Recall@10
1332
+ - type: dot_ndcg@10
1333
+ value: 0.6020710919940331
1334
+ name: Dot Ndcg@10
1335
+ - type: dot_mrr@10
1336
+ value: 0.5799047619047619
1337
+ name: Dot Mrr@10
1338
+ - type: dot_map@100
1339
+ value: 0.5551340236204781
1340
+ name: Dot Map@100
1341
+ - type: query_active_dims
1342
+ value: 82.45999908447266
1343
+ name: Query Active Dims
1344
+ - type: query_sparsity_ratio
1345
+ value: 0.9972983422094073
1346
+ name: Query Sparsity Ratio
1347
+ - type: corpus_active_dims
1348
+ value: 194.24940490722656
1349
+ name: Corpus Active Dims
1350
+ - type: corpus_sparsity_ratio
1351
+ value: 0.9936357576532591
1352
+ name: Corpus Sparsity Ratio
1353
+ - task:
1354
+ type: sparse-information-retrieval
1355
+ name: Sparse Information Retrieval
1356
+ dataset:
1357
+ name: NanoTouche2020
1358
+ type: NanoTouche2020
1359
+ metrics:
1360
+ - type: dot_accuracy@1
1361
+ value: 0.3877551020408163
1362
+ name: Dot Accuracy@1
1363
+ - type: dot_accuracy@3
1364
+ value: 0.7551020408163265
1365
+ name: Dot Accuracy@3
1366
+ - type: dot_accuracy@5
1367
+ value: 0.8367346938775511
1368
+ name: Dot Accuracy@5
1369
+ - type: dot_accuracy@10
1370
+ value: 0.9591836734693877
1371
+ name: Dot Accuracy@10
1372
+ - type: dot_precision@1
1373
+ value: 0.3877551020408163
1374
+ name: Dot Precision@1
1375
+ - type: dot_precision@3
1376
+ value: 0.4693877551020407
1377
+ name: Dot Precision@3
1378
+ - type: dot_precision@5
1379
+ value: 0.4163265306122449
1380
+ name: Dot Precision@5
1381
+ - type: dot_precision@10
1382
+ value: 0.33877551020408164
1383
+ name: Dot Precision@10
1384
+ - type: dot_recall@1
1385
+ value: 0.02376760958202688
1386
+ name: Dot Recall@1
1387
+ - type: dot_recall@3
1388
+ value: 0.08934182714819683
1389
+ name: Dot Recall@3
1390
+ - type: dot_recall@5
1391
+ value: 0.12879429112534482
1392
+ name: Dot Recall@5
1393
+ - type: dot_recall@10
1394
+ value: 0.21659229068805946
1395
+ name: Dot Recall@10
1396
+ - type: dot_ndcg@10
1397
+ value: 0.37622550913382224
1398
+ name: Dot Ndcg@10
1399
+ - type: dot_mrr@10
1400
+ value: 0.5806689342403627
1401
+ name: Dot Mrr@10
1402
+ - type: dot_map@100
1403
+ value: 0.2560796141253303
1404
+ name: Dot Map@100
1405
+ - type: query_active_dims
1406
+ value: 79.12245178222656
1407
+ name: Query Active Dims
1408
+ - type: query_sparsity_ratio
1409
+ value: 0.9974076911151881
1410
+ name: Query Sparsity Ratio
1411
+ - type: corpus_active_dims
1412
+ value: 135.00782775878906
1413
+ name: Corpus Active Dims
1414
+ - type: corpus_sparsity_ratio
1415
+ value: 0.9955767044178366
1416
+ name: Corpus Sparsity Ratio
1417
+ ---
1418
+
1419
+ # splade-distilbert-base-uncased trained on Quora Duplicates Questions
1420
+
1421
+ This is a [SPLADE Sparse Encoder](https://www.sbert.net/docs/sparse_encoder/usage/usage.html) model finetuned from [distilbert/distilbert-base-uncased](https://huggingface.co/distilbert/distilbert-base-uncased) on the [quora-duplicates](https://huggingface.co/datasets/sentence-transformers/quora-duplicates) dataset using the [sentence-transformers](https://www.SBERT.net) library. It maps sentences & paragraphs to a 30522-dimensional sparse vector space and can be used for semantic search and sparse retrieval.
1422
+ ## Model Details
1423
+
1424
+ ### Model Description
1425
+ - **Model Type:** SPLADE Sparse Encoder
1426
+ - **Base model:** [distilbert/distilbert-base-uncased](https://huggingface.co/distilbert/distilbert-base-uncased) <!-- at revision 12040accade4e8a0f71eabdb258fecc2e7e948be -->
1427
+ - **Maximum Sequence Length:** 256 tokens
1428
+ - **Output Dimensionality:** 30522 dimensions
1429
+ - **Similarity Function:** Dot Product
1430
+ - **Training Dataset:**
1431
+ - [quora-duplicates](https://huggingface.co/datasets/sentence-transformers/quora-duplicates)
1432
+ - **Language:** en
1433
+ - **License:** apache-2.0
1434
+
1435
+ ### Model Sources
1436
+
1437
+ - **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
1438
+ - **Documentation:** [Sparse Encoder Documentation](https://www.sbert.net/docs/sparse_encoder/usage/usage.html)
1439
+ - **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
1440
+ - **Hugging Face:** [Sparse Encoders on Hugging Face](https://huggingface.co/models?library=sentence-transformers&other=sparse-encoder)
1441
+
1442
+ ### Full Model Architecture
1443
+
1444
+ ```
1445
+ SparseEncoder(
1446
+ (0): MLMTransformer({'max_seq_length': 256, 'do_lower_case': False}) with MLMTransformer model: DistilBertForMaskedLM
1447
+ (1): SpladePooling({'pooling_strategy': 'max', 'activation_function': 'relu', 'word_embedding_dimension': 30522})
1448
+ )
1449
+ ```
1450
+
1451
+ ## Usage
1452
+
1453
+ ### Direct Usage (Sentence Transformers)
1454
+
1455
+ First install the Sentence Transformers library:
1456
+
1457
+ ```bash
1458
+ pip install -U sentence-transformers
1459
+ ```
1460
+
1461
+ Then you can load this model and run inference.
1462
+ ```python
1463
+ from sentence_transformers import SparseEncoder
1464
+
1465
+ # Download from the 🤗 Hub
1466
+ model = SparseEncoder("arthurbresnu/splade-distilbert-base-uncased-quora-duplicates")
1467
+ # Run inference
1468
+ sentences = [
1469
+ 'What accomplishments did Hillary Clinton achieve during her time as Secretary of State?',
1470
+ "What are Hillary Clinton's most recognized accomplishments while Secretary of State?",
1471
+ 'What are Hillary Clinton’s qualifications to be President?',
1472
+ ]
1473
+ embeddings = model.encode(sentences)
1474
+ print(embeddings.shape)
1475
+ # (3, 30522)
1476
+
1477
+ # Get the similarity scores for the embeddings
1478
+ similarities = model.similarity(embeddings, embeddings)
1479
+ print(similarities.shape)
1480
+ # [3, 3]
1481
+ ```
1482
+
1483
+ <!--
1484
+ ### Direct Usage (Transformers)
1485
+
1486
+ <details><summary>Click to see the direct usage in Transformers</summary>
1487
+
1488
+ </details>
1489
+ -->
1490
+
1491
+ <!--
1492
+ ### Downstream Usage (Sentence Transformers)
1493
+
1494
+ You can finetune this model on your own dataset.
1495
+
1496
+ <details><summary>Click to expand</summary>
1497
+
1498
+ </details>
1499
+ -->
1500
+
1501
+ <!--
1502
+ ### Out-of-Scope Use
1503
+
1504
+ *List how the model may foreseeably be misused and address what users ought not to do with the model.*
1505
+ -->
1506
+
1507
+ ## Evaluation
1508
+
1509
+ ### Metrics
1510
+
1511
+ #### Sparse Binary Classification
1512
+
1513
+ * Dataset: `quora_duplicates_dev`
1514
+ * Evaluated with [<code>SparseBinaryClassificationEvaluator</code>](https://sbert.net/docs/package_reference/sparse_encoder/evaluation.html#sentence_transformers.sparse_encoder.evaluation.SparseBinaryClassificationEvaluator)
1515
+
1516
+ | Metric | Value |
1517
+ |:-----------------------------|:-----------|
1518
+ | cosine_accuracy | 0.758 |
1519
+ | cosine_accuracy_threshold | 0.8166 |
1520
+ | cosine_f1 | 0.6793 |
1521
+ | cosine_f1_threshold | 0.5696 |
1522
+ | cosine_precision | 0.5488 |
1523
+ | cosine_recall | 0.8913 |
1524
+ | cosine_ap | 0.6888 |
1525
+ | cosine_mcc | 0.5082 |
1526
+ | dot_accuracy | 0.765 |
1527
+ | dot_accuracy_threshold | 51.6699 |
1528
+ | dot_f1 | 0.6762 |
1529
+ | dot_f1_threshold | 46.5249 |
1530
+ | dot_precision | 0.5817 |
1531
+ | dot_recall | 0.8075 |
1532
+ | dot_ap | 0.6336 |
1533
+ | dot_mcc | 0.4996 |
1534
+ | euclidean_accuracy | 0.677 |
1535
+ | euclidean_accuracy_threshold | -14.2724 |
1536
+ | euclidean_f1 | 0.486 |
1537
+ | euclidean_f1_threshold | -0.6445 |
1538
+ | euclidean_precision | 0.3213 |
1539
+ | euclidean_recall | 0.9969 |
1540
+ | euclidean_ap | 0.2033 |
1541
+ | euclidean_mcc | -0.0459 |
1542
+ | manhattan_accuracy | 0.677 |
1543
+ | manhattan_accuracy_threshold | -161.7768 |
1544
+ | manhattan_f1 | 0.486 |
1545
+ | manhattan_f1_threshold | -3.0495 |
1546
+ | manhattan_precision | 0.3213 |
1547
+ | manhattan_recall | 0.9969 |
1548
+ | manhattan_ap | 0.2044 |
1549
+ | manhattan_mcc | -0.0459 |
1550
+ | max_accuracy | 0.765 |
1551
+ | max_accuracy_threshold | 51.6699 |
1552
+ | max_f1 | 0.6793 |
1553
+ | max_f1_threshold | 46.5249 |
1554
+ | max_precision | 0.5817 |
1555
+ | max_recall | 0.9969 |
1556
+ | **max_ap** | **0.6888** |
1557
+ | max_mcc | 0.5082 |
1558
+ | active_dims | 78.3228 |
1559
+ | sparsity_ratio | 0.9974 |
1560
+
1561
+ #### Sparse Information Retrieval
1562
+
1563
+ * Datasets: `NanoMSMARCO`, `NanoNQ`, `NanoNFCorpus`, `NanoQuoraRetrieval`, `NanoClimateFEVER`, `NanoDBPedia`, `NanoFEVER`, `NanoFiQA2018`, `NanoHotpotQA`, `NanoMSMARCO`, `NanoNFCorpus`, `NanoNQ`, `NanoQuoraRetrieval`, `NanoSCIDOCS`, `NanoArguAna`, `NanoSciFact` and `NanoTouche2020`
1564
+ * Evaluated with [<code>SparseInformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sparse_encoder/evaluation.html#sentence_transformers.sparse_encoder.evaluation.SparseInformationRetrievalEvaluator)
1565
+
1566
+ | Metric | NanoMSMARCO | NanoNQ | NanoNFCorpus | NanoQuoraRetrieval | NanoClimateFEVER | NanoDBPedia | NanoFEVER | NanoFiQA2018 | NanoHotpotQA | NanoSCIDOCS | NanoArguAna | NanoSciFact | NanoTouche2020 |
1567
+ |:----------------------|:------------|:-----------|:-------------|:-------------------|:-----------------|:------------|:----------|:-------------|:-------------|:------------|:------------|:------------|:---------------|
1568
+ | dot_accuracy@1 | 0.22 | 0.38 | 0.34 | 0.94 | 0.18 | 0.6 | 0.58 | 0.28 | 0.78 | 0.36 | 0.1 | 0.52 | 0.3878 |
1569
+ | dot_accuracy@3 | 0.42 | 0.54 | 0.5 | 0.98 | 0.32 | 0.84 | 0.76 | 0.42 | 0.84 | 0.58 | 0.34 | 0.62 | 0.7551 |
1570
+ | dot_accuracy@5 | 0.52 | 0.62 | 0.54 | 0.98 | 0.4 | 0.84 | 0.8 | 0.46 | 0.92 | 0.68 | 0.42 | 0.64 | 0.8367 |
1571
+ | dot_accuracy@10 | 0.76 | 0.62 | 0.58 | 0.98 | 0.48 | 0.92 | 0.86 | 0.5 | 0.98 | 0.76 | 0.44 | 0.76 | 0.9592 |
1572
+ | dot_precision@1 | 0.22 | 0.38 | 0.34 | 0.94 | 0.18 | 0.6 | 0.58 | 0.28 | 0.78 | 0.36 | 0.1 | 0.52 | 0.3878 |
1573
+ | dot_precision@3 | 0.14 | 0.18 | 0.3067 | 0.3933 | 0.1067 | 0.5267 | 0.2667 | 0.18 | 0.3733 | 0.2733 | 0.1133 | 0.2133 | 0.4694 |
1574
+ | dot_precision@5 | 0.104 | 0.124 | 0.26 | 0.248 | 0.084 | 0.456 | 0.168 | 0.136 | 0.284 | 0.212 | 0.084 | 0.14 | 0.4163 |
1575
+ | dot_precision@10 | 0.076 | 0.064 | 0.198 | 0.132 | 0.054 | 0.422 | 0.09 | 0.084 | 0.16 | 0.152 | 0.044 | 0.084 | 0.3388 |
1576
+ | dot_recall@1 | 0.22 | 0.36 | 0.0116 | 0.8173 | 0.085 | 0.0457 | 0.5467 | 0.1418 | 0.39 | 0.0757 | 0.1 | 0.475 | 0.0238 |
1577
+ | dot_recall@3 | 0.42 | 0.52 | 0.0606 | 0.928 | 0.1467 | 0.1537 | 0.7467 | 0.2429 | 0.56 | 0.1697 | 0.34 | 0.58 | 0.0893 |
1578
+ | dot_recall@5 | 0.52 | 0.6 | 0.0826 | 0.946 | 0.1783 | 0.1908 | 0.7867 | 0.2772 | 0.71 | 0.2177 | 0.42 | 0.615 | 0.1288 |
1579
+ | dot_recall@10 | 0.76 | 0.61 | 0.098 | 0.97 | 0.215 | 0.2936 | 0.8467 | 0.3289 | 0.8 | 0.3107 | 0.44 | 0.74 | 0.2166 |
1580
+ | **dot_ndcg@10** | **0.4532** | **0.4828** | **0.2467** | **0.9467** | **0.1845** | **0.5071** | **0.707** | **0.2881** | **0.7143** | **0.3029** | **0.2782** | **0.6021** | **0.3762** |
1581
+ | dot_mrr@10 | 0.3601 | 0.4537 | 0.422 | 0.96 | 0.2674 | 0.7147 | 0.6765 | 0.3562 | 0.8362 | 0.4944 | 0.2247 | 0.5799 | 0.5807 |
1582
+ | dot_map@100 | 0.3733 | 0.4454 | 0.094 | 0.9291 | 0.1476 | 0.3907 | 0.6651 | 0.2415 | 0.6181 | 0.2167 | 0.2333 | 0.5551 | 0.2561 |
1583
+ | query_active_dims | 74.76 | 74.74 | 79.7 | 76.58 | 89.86 | 69.52 | 89.88 | 82.86 | 91.98 | 94.3 | 189.1 | 82.46 | 79.1225 |
1584
+ | query_sparsity_ratio | 0.9976 | 0.9976 | 0.9974 | 0.9975 | 0.9971 | 0.9977 | 0.9971 | 0.9973 | 0.997 | 0.9969 | 0.9938 | 0.9973 | 0.9974 |
1585
+ | corpus_active_dims | 103.0652 | 141.3148 | 202.1727 | 77.5906 | 221.7553 | 135.9335 | 221.2156 | 130.937 | 152.0157 | 199.6463 | 164.0333 | 194.2494 | 135.0078 |
1586
+ | corpus_sparsity_ratio | 0.9966 | 0.9954 | 0.9934 | 0.9975 | 0.9927 | 0.9955 | 0.9928 | 0.9957 | 0.995 | 0.9935 | 0.9946 | 0.9936 | 0.9956 |
1587
+
1588
+ #### Sparse Nano BEIR
1589
+
1590
+ * Dataset: `NanoBEIR_mean`
1591
+ * Evaluated with [<code>SparseNanoBEIREvaluator</code>](https://sbert.net/docs/package_reference/sparse_encoder/evaluation.html#sentence_transformers.sparse_encoder.evaluation.SparseNanoBEIREvaluator) with these parameters:
1592
+ ```json
1593
+ {
1594
+ "dataset_names": [
1595
+ "msmarco",
1596
+ "nq",
1597
+ "nfcorpus",
1598
+ "quoraretrieval"
1599
+ ]
1600
+ }
1601
+ ```
1602
+
1603
+ | Metric | Value |
1604
+ |:----------------------|:-----------|
1605
+ | dot_accuracy@1 | 0.47 |
1606
+ | dot_accuracy@3 | 0.61 |
1607
+ | dot_accuracy@5 | 0.665 |
1608
+ | dot_accuracy@10 | 0.735 |
1609
+ | dot_precision@1 | 0.47 |
1610
+ | dot_precision@3 | 0.255 |
1611
+ | dot_precision@5 | 0.184 |
1612
+ | dot_precision@10 | 0.1175 |
1613
+ | dot_recall@1 | 0.3522 |
1614
+ | dot_recall@3 | 0.4821 |
1615
+ | dot_recall@5 | 0.5372 |
1616
+ | dot_recall@10 | 0.6095 |
1617
+ | **dot_ndcg@10** | **0.5324** |
1618
+ | dot_mrr@10 | 0.5489 |
1619
+ | dot_map@100 | 0.4605 |
1620
+ | query_active_dims | 76.445 |
1621
+ | query_sparsity_ratio | 0.9975 |
1622
+ | corpus_active_dims | 122.7978 |
1623
+ | corpus_sparsity_ratio | 0.996 |
1624
+
1625
+ #### Sparse Nano BEIR
1626
+
1627
+ * Dataset: `NanoBEIR_mean`
1628
+ * Evaluated with [<code>SparseNanoBEIREvaluator</code>](https://sbert.net/docs/package_reference/sparse_encoder/evaluation.html#sentence_transformers.sparse_encoder.evaluation.SparseNanoBEIREvaluator) with these parameters:
1629
+ ```json
1630
+ {
1631
+ "dataset_names": [
1632
+ "climatefever",
1633
+ "dbpedia",
1634
+ "fever",
1635
+ "fiqa2018",
1636
+ "hotpotqa",
1637
+ "msmarco",
1638
+ "nfcorpus",
1639
+ "nq",
1640
+ "quoraretrieval",
1641
+ "scidocs",
1642
+ "arguana",
1643
+ "scifact",
1644
+ "touche2020"
1645
+ ]
1646
+ }
1647
+ ```
1648
+
1649
+ | Metric | Value |
1650
+ |:----------------------|:-----------|
1651
+ | dot_accuracy@1 | 0.436 |
1652
+ | dot_accuracy@3 | 0.6089 |
1653
+ | dot_accuracy@5 | 0.6659 |
1654
+ | dot_accuracy@10 | 0.7384 |
1655
+ | dot_precision@1 | 0.436 |
1656
+ | dot_precision@3 | 0.2725 |
1657
+ | dot_precision@5 | 0.2089 |
1658
+ | dot_precision@10 | 0.1461 |
1659
+ | dot_recall@1 | 0.2533 |
1660
+ | dot_recall@3 | 0.3813 |
1661
+ | dot_recall@5 | 0.4364 |
1662
+ | dot_recall@10 | 0.51 |
1663
+ | **dot_ndcg@10** | **0.4685** |
1664
+ | dot_mrr@10 | 0.5328 |
1665
+ | dot_map@100 | 0.3974 |
1666
+ | query_active_dims | 90.3914 |
1667
+ | query_sparsity_ratio | 0.997 |
1668
+ | corpus_active_dims | 152.3669 |
1669
+ | corpus_sparsity_ratio | 0.995 |
1670
+
1671
+ <!--
1672
+ ## Bias, Risks and Limitations
1673
+
1674
+ *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
1675
+ -->
1676
+
1677
+ <!--
1678
+ ### Recommendations
1679
+
1680
+ *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
1681
+ -->
1682
+
1683
+ ## Training Details
1684
+
1685
+ ### Training Dataset
1686
+
1687
+ #### quora-duplicates
1688
+
1689
+ * Dataset: [quora-duplicates](https://huggingface.co/datasets/sentence-transformers/quora-duplicates) at [451a485](https://huggingface.co/datasets/sentence-transformers/quora-duplicates/tree/451a4850bd141edb44ade1b5828c259abd762cdb)
1690
+ * Size: 99,000 training samples
1691
+ * Columns: <code>anchor</code>, <code>positive</code>, and <code>negative</code>
1692
+ * Approximate statistics based on the first 1000 samples:
1693
+ | | anchor | positive | negative |
1694
+ |:--------|:---------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|
1695
+ | type | string | string | string |
1696
+ | details | <ul><li>min: 6 tokens</li><li>mean: 14.1 tokens</li><li>max: 39 tokens</li></ul> | <ul><li>min: 6 tokens</li><li>mean: 13.83 tokens</li><li>max: 41 tokens</li></ul> | <ul><li>min: 6 tokens</li><li>mean: 15.21 tokens</li><li>max: 75 tokens</li></ul> |
1697
+ * Samples:
1698
+ | anchor | positive | negative |
1699
+ |:----------------------------------------------------------------------|:---------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
1700
+ | <code>What are the best GMAT coaching institutes in Delhi NCR?</code> | <code>Which are the best GMAT coaching institutes in Delhi/NCR?</code> | <code>What are the best GMAT coaching institutes in Delhi-Noida Area?</code> |
1701
+ | <code>Is a third world war coming?</code> | <code>Is World War 3 more imminent than expected?</code> | <code>Since the UN is unable to control terrorism and groups like ISIS, al-Qaeda and countries that promote terrorism (even though it consumed those countries), can we assume that the world is heading towards World War III?</code> |
1702
+ | <code>Should I build iOS or Android apps first?</code> | <code>Should people choose Android or iOS first to build their App?</code> | <code>How much more effort is it to build your app on both iOS and Android?</code> |
1703
+ * Loss: [<code>SpladeLoss</code>](https://sbert.net/docs/package_reference/sparse_encoder/losses.html#spladeloss) with these parameters:
1704
+ ```json
1705
+ {
1706
+ "loss": "SparseMultipleNegativesRankingLoss(scale=1.0, similarity_fct='dot_score')",
1707
+ "lambda_corpus": 3e-05,
1708
+ "lambda_query": 5e-05
1709
+ }
1710
+ ```
1711
+
1712
+ ### Evaluation Dataset
1713
+
1714
+ #### quora-duplicates
1715
+
1716
+ * Dataset: [quora-duplicates](https://huggingface.co/datasets/sentence-transformers/quora-duplicates) at [451a485](https://huggingface.co/datasets/sentence-transformers/quora-duplicates/tree/451a4850bd141edb44ade1b5828c259abd762cdb)
1717
+ * Size: 1,000 evaluation samples
1718
+ * Columns: <code>anchor</code>, <code>positive</code>, and <code>negative</code>
1719
+ * Approximate statistics based on the first 1000 samples:
1720
+ | | anchor | positive | negative |
1721
+ |:--------|:----------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|
1722
+ | type | string | string | string |
1723
+ | details | <ul><li>min: 6 tokens</li><li>mean: 14.05 tokens</li><li>max: 40 tokens</li></ul> | <ul><li>min: 6 tokens</li><li>mean: 14.14 tokens</li><li>max: 44 tokens</li></ul> | <ul><li>min: 6 tokens</li><li>mean: 14.56 tokens</li><li>max: 60 tokens</li></ul> |
1724
+ * Samples:
1725
+ | anchor | positive | negative |
1726
+ |:-------------------------------------------------------------------|:------------------------------------------------------------|:-----------------------------------------------------------------|
1727
+ | <code>What happens if we use petrol in diesel vehicles?</code> | <code>Why can't we use petrol in diesel?</code> | <code>Why are diesel engines noisier than petrol engines?</code> |
1728
+ | <code>Why is Saltwater taffy candy imported in Switzerland?</code> | <code>Why is Saltwater taffy candy imported in Laos?</code> | <code>Is salt a consumer product?</code> |
1729
+ | <code>Which is your favourite film in 2016?</code> | <code>What movie is the best movie of 2016?</code> | <code>What will the best movie of 2017 be?</code> |
1730
+ * Loss: [<code>SpladeLoss</code>](https://sbert.net/docs/package_reference/sparse_encoder/losses.html#spladeloss) with these parameters:
1731
+ ```json
1732
+ {
1733
+ "loss": "SparseMultipleNegativesRankingLoss(scale=1.0, similarity_fct='dot_score')",
1734
+ "lambda_corpus": 3e-05,
1735
+ "lambda_query": 5e-05
1736
+ }
1737
+ ```
1738
+
1739
+ ### Training Hyperparameters
1740
+ #### Non-Default Hyperparameters
1741
+
1742
+ - `eval_strategy`: steps
1743
+ - `per_device_train_batch_size`: 12
1744
+ - `per_device_eval_batch_size`: 12
1745
+ - `learning_rate`: 2e-05
1746
+ - `num_train_epochs`: 1
1747
+ - `bf16`: True
1748
+ - `load_best_model_at_end`: True
1749
+ - `batch_sampler`: no_duplicates
1750
+
1751
+ #### All Hyperparameters
1752
+ <details><summary>Click to expand</summary>
1753
+
1754
+ - `overwrite_output_dir`: False
1755
+ - `do_predict`: False
1756
+ - `eval_strategy`: steps
1757
+ - `prediction_loss_only`: True
1758
+ - `per_device_train_batch_size`: 12
1759
+ - `per_device_eval_batch_size`: 12
1760
+ - `per_gpu_train_batch_size`: None
1761
+ - `per_gpu_eval_batch_size`: None
1762
+ - `gradient_accumulation_steps`: 1
1763
+ - `eval_accumulation_steps`: None
1764
+ - `torch_empty_cache_steps`: None
1765
+ - `learning_rate`: 2e-05
1766
+ - `weight_decay`: 0.0
1767
+ - `adam_beta1`: 0.9
1768
+ - `adam_beta2`: 0.999
1769
+ - `adam_epsilon`: 1e-08
1770
+ - `max_grad_norm`: 1.0
1771
+ - `num_train_epochs`: 1
1772
+ - `max_steps`: -1
1773
+ - `lr_scheduler_type`: linear
1774
+ - `lr_scheduler_kwargs`: {}
1775
+ - `warmup_ratio`: 0.0
1776
+ - `warmup_steps`: 0
1777
+ - `log_level`: passive
1778
+ - `log_level_replica`: warning
1779
+ - `log_on_each_node`: True
1780
+ - `logging_nan_inf_filter`: True
1781
+ - `save_safetensors`: True
1782
+ - `save_on_each_node`: False
1783
+ - `save_only_model`: False
1784
+ - `restore_callback_states_from_checkpoint`: False
1785
+ - `no_cuda`: False
1786
+ - `use_cpu`: False
1787
+ - `use_mps_device`: False
1788
+ - `seed`: 42
1789
+ - `data_seed`: None
1790
+ - `jit_mode_eval`: False
1791
+ - `use_ipex`: False
1792
+ - `bf16`: True
1793
+ - `fp16`: False
1794
+ - `fp16_opt_level`: O1
1795
+ - `half_precision_backend`: auto
1796
+ - `bf16_full_eval`: False
1797
+ - `fp16_full_eval`: False
1798
+ - `tf32`: None
1799
+ - `local_rank`: 0
1800
+ - `ddp_backend`: None
1801
+ - `tpu_num_cores`: None
1802
+ - `tpu_metrics_debug`: False
1803
+ - `debug`: []
1804
+ - `dataloader_drop_last`: False
1805
+ - `dataloader_num_workers`: 0
1806
+ - `dataloader_prefetch_factor`: None
1807
+ - `past_index`: -1
1808
+ - `disable_tqdm`: False
1809
+ - `remove_unused_columns`: True
1810
+ - `label_names`: None
1811
+ - `load_best_model_at_end`: True
1812
+ - `ignore_data_skip`: False
1813
+ - `fsdp`: []
1814
+ - `fsdp_min_num_params`: 0
1815
+ - `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
1816
+ - `tp_size`: 0
1817
+ - `fsdp_transformer_layer_cls_to_wrap`: None
1818
+ - `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
1819
+ - `deepspeed`: None
1820
+ - `label_smoothing_factor`: 0.0
1821
+ - `optim`: adamw_torch
1822
+ - `optim_args`: None
1823
+ - `adafactor`: False
1824
+ - `group_by_length`: False
1825
+ - `length_column_name`: length
1826
+ - `ddp_find_unused_parameters`: None
1827
+ - `ddp_bucket_cap_mb`: None
1828
+ - `ddp_broadcast_buffers`: False
1829
+ - `dataloader_pin_memory`: True
1830
+ - `dataloader_persistent_workers`: False
1831
+ - `skip_memory_metrics`: True
1832
+ - `use_legacy_prediction_loop`: False
1833
+ - `push_to_hub`: False
1834
+ - `resume_from_checkpoint`: None
1835
+ - `hub_model_id`: None
1836
+ - `hub_strategy`: every_save
1837
+ - `hub_private_repo`: None
1838
+ - `hub_always_push`: False
1839
+ - `gradient_checkpointing`: False
1840
+ - `gradient_checkpointing_kwargs`: None
1841
+ - `include_inputs_for_metrics`: False
1842
+ - `include_for_metrics`: []
1843
+ - `eval_do_concat_batches`: True
1844
+ - `fp16_backend`: auto
1845
+ - `push_to_hub_model_id`: None
1846
+ - `push_to_hub_organization`: None
1847
+ - `mp_parameters`:
1848
+ - `auto_find_batch_size`: False
1849
+ - `full_determinism`: False
1850
+ - `torchdynamo`: None
1851
+ - `ray_scope`: last
1852
+ - `ddp_timeout`: 1800
1853
+ - `torch_compile`: False
1854
+ - `torch_compile_backend`: None
1855
+ - `torch_compile_mode`: None
1856
+ - `dispatch_batches`: None
1857
+ - `split_batches`: None
1858
+ - `include_tokens_per_second`: False
1859
+ - `include_num_input_tokens_seen`: False
1860
+ - `neftune_noise_alpha`: None
1861
+ - `optim_target_modules`: None
1862
+ - `batch_eval_metrics`: False
1863
+ - `eval_on_start`: False
1864
+ - `use_liger_kernel`: False
1865
+ - `eval_use_gather_object`: False
1866
+ - `average_tokens_across_devices`: False
1867
+ - `prompts`: None
1868
+ - `batch_sampler`: no_duplicates
1869
+ - `multi_dataset_batch_sampler`: proportional
1870
+
1871
+ </details>
1872
+
1873
+ ### Training Logs
1874
+ | Epoch | Step | Training Loss | Validation Loss | quora_duplicates_dev_max_ap | NanoMSMARCO_dot_ndcg@10 | NanoNQ_dot_ndcg@10 | NanoNFCorpus_dot_ndcg@10 | NanoQuoraRetrieval_dot_ndcg@10 | NanoBEIR_mean_dot_ndcg@10 | NanoClimateFEVER_dot_ndcg@10 | NanoDBPedia_dot_ndcg@10 | NanoFEVER_dot_ndcg@10 | NanoFiQA2018_dot_ndcg@10 | NanoHotpotQA_dot_ndcg@10 | NanoSCIDOCS_dot_ndcg@10 | NanoArguAna_dot_ndcg@10 | NanoSciFact_dot_ndcg@10 | NanoTouche2020_dot_ndcg@10 |
1875
+ |:-------:|:--------:|:-------------:|:---------------:|:---------------------------:|:-----------------------:|:------------------:|:------------------------:|:------------------------------:|:-------------------------:|:----------------------------:|:-----------------------:|:---------------------:|:------------------------:|:------------------------:|:-----------------------:|:-----------------------:|:-----------------------:|:--------------------------:|
1876
+ | 0.0242 | 200 | 8.3389 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1877
+ | 0.0485 | 400 | 0.4397 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1878
+ | 0.0727 | 600 | 0.3737 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1879
+ | 0.0970 | 800 | 0.2666 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1880
+ | 0.1212 | 1000 | 0.288 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1881
+ | 0.1455 | 1200 | 0.1977 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1882
+ | 0.1697 | 1400 | 0.2707 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1883
+ | 0.1939 | 1600 | 0.1951 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1884
+ | 0.2 | 1650 | - | 0.1669 | 0.6472 | 0.3052 | 0.2793 | 0.1711 | 0.9281 | 0.4209 | - | - | - | - | - | - | - | - | - |
1885
+ | 0.2182 | 1800 | 0.2178 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1886
+ | 0.2424 | 2000 | 0.2174 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1887
+ | 0.2667 | 2200 | 0.1832 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1888
+ | 0.2909 | 2400 | 0.1879 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1889
+ | 0.3152 | 2600 | 0.1723 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1890
+ | 0.3394 | 2800 | 0.1543 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1891
+ | 0.3636 | 3000 | 0.1559 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1892
+ | 0.3879 | 3200 | 0.1575 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1893
+ | 0.4 | 3300 | - | 0.1149 | 0.6749 | 0.3894 | 0.4467 | 0.2360 | 0.9292 | 0.5003 | - | - | - | - | - | - | - | - | - |
1894
+ | 0.4121 | 3400 | 0.1395 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1895
+ | 0.4364 | 3600 | 0.1596 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1896
+ | 0.4606 | 3800 | 0.1595 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1897
+ | 0.4848 | 4000 | 0.1211 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1898
+ | 0.5091 | 4200 | 0.1163 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1899
+ | 0.5333 | 4400 | 0.1182 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1900
+ | 0.5576 | 4600 | 0.1337 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1901
+ | 0.5818 | 4800 | 0.1362 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1902
+ | 0.6 | 4950 | - | 0.1001 | 0.6802 | 0.4093 | 0.4269 | 0.2341 | 0.9365 | 0.5017 | - | - | - | - | - | - | - | - | - |
1903
+ | 0.6061 | 5000 | 0.1112 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1904
+ | 0.6303 | 5200 | 0.1064 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1905
+ | 0.6545 | 5400 | 0.119 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1906
+ | 0.6788 | 5600 | 0.1077 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1907
+ | 0.7030 | 5800 | 0.1398 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1908
+ | 0.7273 | 6000 | 0.09 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1909
+ | 0.7515 | 6200 | 0.0903 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1910
+ | 0.7758 | 6400 | 0.1082 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1911
+ | 0.8 | 6600 | 0.1122 | 0.0901 | 0.6941 | 0.4451 | 0.4757 | 0.2542 | 0.9411 | 0.5290 | - | - | - | - | - | - | - | - | - |
1912
+ | 0.8242 | 6800 | 0.0708 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1913
+ | 0.8485 | 7000 | 0.1291 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1914
+ | 0.8727 | 7200 | 0.1165 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1915
+ | 0.8970 | 7400 | 0.0735 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1916
+ | 0.9212 | 7600 | 0.0775 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1917
+ | 0.9455 | 7800 | 0.0945 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1918
+ | 0.9697 | 8000 | 0.0912 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1919
+ | 0.9939 | 8200 | 0.104 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1920
+ | **1.0** | **8250** | **-** | **0.0686** | **0.6888** | **0.4532** | **0.4828** | **0.2467** | **0.9467** | **0.5324** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** |
1921
+ | -1 | -1 | - | - | - | 0.4532 | 0.4828 | 0.2467 | 0.9467 | 0.4685 | 0.1845 | 0.5071 | 0.7070 | 0.2881 | 0.7143 | 0.3029 | 0.2782 | 0.6021 | 0.3762 |
1922
+
1923
+ * The bold row denotes the saved checkpoint.
1924
+
1925
+ ### Environmental Impact
1926
+ Carbon emissions were measured using [CodeCarbon](https://github.com/mlco2/codecarbon).
1927
+ - **Energy Consumed**: 0.025 kWh
1928
+ - **Carbon Emitted**: 0.001 kg of CO2
1929
+ - **Hours Used**: 0.222 hours
1930
+
1931
+ ### Training Hardware
1932
+ - **On Cloud**: No
1933
+ - **GPU Model**: 1 x NVIDIA GeForce RTX 3070 Ti Laptop GPU
1934
+ - **CPU Model**: AMD Ryzen 9 6900HX with Radeon Graphics
1935
+ - **RAM Size**: 30.61 GB
1936
+
1937
+ ### Framework Versions
1938
+ - Python: 3.12.9
1939
+ - Sentence Transformers: 4.2.0.dev0
1940
+ - Transformers: 4.50.3
1941
+ - PyTorch: 2.6.0+cu124
1942
+ - Accelerate: 1.6.0
1943
+ - Datasets: 3.5.0
1944
+ - Tokenizers: 0.21.1
1945
+
1946
+ ## Citation
1947
+
1948
+ ### BibTeX
1949
+
1950
+ #### Sentence Transformers
1951
+ ```bibtex
1952
+ @inproceedings{reimers-2019-sentence-bert,
1953
+ title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
1954
+ author = "Reimers, Nils and Gurevych, Iryna",
1955
+ booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
1956
+ month = "11",
1957
+ year = "2019",
1958
+ publisher = "Association for Computational Linguistics",
1959
+ url = "https://arxiv.org/abs/1908.10084",
1960
+ }
1961
+ ```
1962
+
1963
+ #### SpladeLoss
1964
+ ```bibtex
1965
+ @misc{formal2022distillationhardnegativesampling,
1966
+ title={From Distillation to Hard Negative Sampling: Making Sparse Neural IR Models More Effective},
1967
+ author={Thibault Formal and Carlos Lassance and Benjamin Piwowarski and Stéphane Clinchant},
1968
+ year={2022},
1969
+ eprint={2205.04733},
1970
+ archivePrefix={arXiv},
1971
+ primaryClass={cs.IR},
1972
+ url={https://arxiv.org/abs/2205.04733},
1973
+ }
1974
+ ```
1975
+
1976
+ #### SparseMultipleNegativesRankingLoss
1977
+ ```bibtex
1978
+ @misc{henderson2017efficient,
1979
+ title={Efficient Natural Language Response Suggestion for Smart Reply},
1980
+ author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
1981
+ year={2017},
1982
+ eprint={1705.00652},
1983
+ archivePrefix={arXiv},
1984
+ primaryClass={cs.CL}
1985
+ }
1986
+ ```
1987
+
1988
+ #### FlopsLoss
1989
+ ```bibtex
1990
+ @article{paria2020minimizing,
1991
+ title={Minimizing flops to learn efficient sparse representations},
1992
+ author={Paria, Biswajit and Yeh, Chih-Kuan and Yen, Ian EH and Xu, Ning and Ravikumar, Pradeep and P{'o}czos, Barnab{'a}s},
1993
+ journal={arXiv preprint arXiv:2004.05665},
1994
+ year={2020}
1995
+ }
1996
+ ```
1997
+
1998
+ <!--
1999
+ ## Glossary
2000
+
2001
+ *Clearly define terms in order to be accessible across audiences.*
2002
+ -->
2003
+
2004
+ <!--
2005
+ ## Model Card Authors
2006
+
2007
+ *Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
2008
+ -->
2009
+
2010
+ <!--
2011
+ ## Model Card Contact
2012
+
2013
+ *Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
2014
+ -->
config.json ADDED
@@ -0,0 +1,23 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "activation": "gelu",
3
+ "architectures": [
4
+ "DistilBertForMaskedLM"
5
+ ],
6
+ "attention_dropout": 0.1,
7
+ "dim": 768,
8
+ "dropout": 0.1,
9
+ "hidden_dim": 3072,
10
+ "initializer_range": 0.02,
11
+ "max_position_embeddings": 512,
12
+ "model_type": "distilbert",
13
+ "n_heads": 12,
14
+ "n_layers": 6,
15
+ "pad_token_id": 0,
16
+ "qa_dropout": 0.1,
17
+ "seq_classif_dropout": 0.2,
18
+ "sinusoidal_pos_embds": false,
19
+ "tie_weights_": true,
20
+ "torch_dtype": "float32",
21
+ "transformers_version": "4.50.3",
22
+ "vocab_size": 30522
23
+ }
config_sentence_transformers.json ADDED
@@ -0,0 +1,11 @@
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "model_type": "SparseEncoder",
3
+ "__version__": {
4
+ "sentence_transformers": "4.2.0.dev0",
5
+ "transformers": "4.50.3",
6
+ "pytorch": "2.6.0+cu124"
7
+ },
8
+ "prompts": {},
9
+ "default_prompt_name": null,
10
+ "similarity_fn_name": "dot"
11
+ }
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e450cb6daf517870f7376c395db7dbdc955ea2445888e66ffe642a1fca1e7d49
3
+ size 267954768
modules.json ADDED
@@ -0,0 +1,14 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ [
2
+ {
3
+ "idx": 0,
4
+ "name": "0",
5
+ "path": "",
6
+ "type": "sentence_transformers.sparse_encoder.models.MLMTransformer"
7
+ },
8
+ {
9
+ "idx": 1,
10
+ "name": "1",
11
+ "path": "1_SpladePooling",
12
+ "type": "sentence_transformers.sparse_encoder.models.SpladePooling"
13
+ }
14
+ ]
sentence_bert_config.json ADDED
@@ -0,0 +1,4 @@
 
 
 
 
 
1
+ {
2
+ "max_seq_length": 256,
3
+ "do_lower_case": false
4
+ }
special_tokens_map.json ADDED
@@ -0,0 +1,7 @@
 
 
 
 
 
 
 
 
1
+ {
2
+ "cls_token": "[CLS]",
3
+ "mask_token": "[MASK]",
4
+ "pad_token": "[PAD]",
5
+ "sep_token": "[SEP]",
6
+ "unk_token": "[UNK]"
7
+ }
tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
tokenizer_config.json ADDED
@@ -0,0 +1,56 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "added_tokens_decoder": {
3
+ "0": {
4
+ "content": "[PAD]",
5
+ "lstrip": false,
6
+ "normalized": false,
7
+ "rstrip": false,
8
+ "single_word": false,
9
+ "special": true
10
+ },
11
+ "100": {
12
+ "content": "[UNK]",
13
+ "lstrip": false,
14
+ "normalized": false,
15
+ "rstrip": false,
16
+ "single_word": false,
17
+ "special": true
18
+ },
19
+ "101": {
20
+ "content": "[CLS]",
21
+ "lstrip": false,
22
+ "normalized": false,
23
+ "rstrip": false,
24
+ "single_word": false,
25
+ "special": true
26
+ },
27
+ "102": {
28
+ "content": "[SEP]",
29
+ "lstrip": false,
30
+ "normalized": false,
31
+ "rstrip": false,
32
+ "single_word": false,
33
+ "special": true
34
+ },
35
+ "103": {
36
+ "content": "[MASK]",
37
+ "lstrip": false,
38
+ "normalized": false,
39
+ "rstrip": false,
40
+ "single_word": false,
41
+ "special": true
42
+ }
43
+ },
44
+ "clean_up_tokenization_spaces": false,
45
+ "cls_token": "[CLS]",
46
+ "do_lower_case": true,
47
+ "extra_special_tokens": {},
48
+ "mask_token": "[MASK]",
49
+ "model_max_length": 512,
50
+ "pad_token": "[PAD]",
51
+ "sep_token": "[SEP]",
52
+ "strip_accents": null,
53
+ "tokenize_chinese_chars": true,
54
+ "tokenizer_class": "DistilBertTokenizer",
55
+ "unk_token": "[UNK]"
56
+ }
vocab.txt ADDED
The diff for this file is too large to render. See raw diff