Update src/tasks_content.py
Browse files- src/tasks_content.py +4 -4
src/tasks_content.py
CHANGED
@@ -24,11 +24,11 @@ TASKS_DESCRIPTIONS = {
|
|
24 |
|
25 |
"ci_builds_repair": """# CI builds repair\n
|
26 |
|
27 |
-
Our CI
|
28 |
-
includes manually curated and assessed
|
29 |
|
30 |
-
We use Pass@1 metric for CI repair.
|
31 |
-
Models can be evaluated in three
|
32 |
* `full` β **no** ground truth diffs are used for model evaluation;
|
33 |
* `oracle: files` β ground truth diffs are used to select files that should be corrected to fix the issue;
|
34 |
* `oracle: files, lines` β ground truth diffs are used to select files and code blocks that should be corrected to fix the issue;
|
|
|
24 |
|
25 |
"ci_builds_repair": """# CI builds repair\n
|
26 |
|
27 |
+
Our CI builds repair benchmark π€ [JetBrains-Research/lca-ci-builds-repair](https://huggingface.co/datasets/JetBrains-Research/lca-ci-builds-repair)
|
28 |
+
includes 77 manually curated and assessed data points coming from 32 Python repositories, which are used to make a model fix a failed build.
|
29 |
|
30 |
+
We use the `Pass@1` metric for CI repair.
|
31 |
+
Models can be evaluated in three types of tasks:
|
32 |
* `full` β **no** ground truth diffs are used for model evaluation;
|
33 |
* `oracle: files` β ground truth diffs are used to select files that should be corrected to fix the issue;
|
34 |
* `oracle: files, lines` β ground truth diffs are used to select files and code blocks that should be corrected to fix the issue;
|