Update README.md
Browse files
README.md
CHANGED
@@ -30,10 +30,10 @@ The model was then evaluated on [Mean Reciprocal Rank (MRR)](https://en.wikipedi
|
|
30 |
When the model has to pick the positive example out of a pool of 32, it almost always ranks it first. When
|
31 |
the pool is significantly enlarged to 10.000 functions, it still ranks the positive example highest most of the time.
|
32 |
|
33 |
-
| Model
|
34 |
-
|
35 |
-
|
|
36 |
-
|
|
37 |
|
38 |
## Purpose and use of the model
|
39 |
|
@@ -62,6 +62,8 @@ either the train or the test set, not both. We have not performed any deduplicat
|
|
62 |
| train | 18,083,285 |
|
63 |
| test | 3,375,741 |
|
64 |
|
|
|
|
|
65 |
### By whom was the dataset collected and annotated?
|
66 |
The dataset was collected by our team. The annotation of similar/non-similar function comes from the different compilation
|
67 |
levels, i.e. what we consider "similar functions" is in fact the same function that has been compiled in a different way.
|
|
|
30 |
When the model has to pick the positive example out of a pool of 32, it almost always ranks it first. When
|
31 |
the pool is significantly enlarged to 10.000 functions, it still ranks the positive example highest most of the time.
|
32 |
|
33 |
+
| Model | Pool size | MRR | Recall@1 |
|
34 |
+
|-----------|-----------|------|----------|
|
35 |
+
| ARM64BERT | 32 | 0.78 | 0.72 |
|
36 |
+
| ARM64BERT | 10.000 | 0.58 | 0.56 |
|
37 |
|
38 |
## Purpose and use of the model
|
39 |
|
|
|
62 |
| train | 18,083,285 |
|
63 |
| test | 3,375,741 |
|
64 |
|
65 |
+
For our training and evaluation code, see our [GitHub repository](https://github.com/NetherlandsForensicInstitute/asmtransformers).
|
66 |
+
|
67 |
### By whom was the dataset collected and annotated?
|
68 |
The dataset was collected by our team. The annotation of similar/non-similar function comes from the different compilation
|
69 |
levels, i.e. what we consider "similar functions" is in fact the same function that has been compiled in a different way.
|