Update src/tasks_content.py
Browse files- src/tasks_content.py +1 -1
src/tasks_content.py
CHANGED
@@ -28,7 +28,7 @@ TASKS_DESCRIPTIONS = {
|
|
28 |
|
29 |
We use Pass@1 metric for CI repair.
|
30 |
Models can be evaluated in three task types:
|
31 |
-
* `full` β
|
32 |
* `oracle: files` β ground truth diffs are used to select files that should be corrected to fix the issue;
|
33 |
* `oracle: files, lines` β ground truth diffs are used to select files and code blocks that should be corrected to fix the issue;
|
34 |
|
|
|
28 |
|
29 |
We use Pass@1 metric for CI repair.
|
30 |
Models can be evaluated in three task types:
|
31 |
+
* `full` β **no** ground truth diffs are used for model evaluation;
|
32 |
* `oracle: files` β ground truth diffs are used to select files that should be corrected to fix the issue;
|
33 |
* `oracle: files, lines` β ground truth diffs are used to select files and code blocks that should be corrected to fix the issue;
|
34 |
|