Areyde commited on
Commit
4b54f71
Β·
verified Β·
1 Parent(s): 2765cea

Update src/tasks_content.py

Browse files
Files changed (1) hide show
  1. src/tasks_content.py +4 -4
src/tasks_content.py CHANGED
@@ -24,11 +24,11 @@ TASKS_DESCRIPTIONS = {
24
 
25
  "ci_builds_repair": """# CI builds repair\n
26
 
27
- Our CI Builds Repair benchmark πŸ€— [JetBrains-Research/lca-ci-builds-repair](https://huggingface.co/datasets/JetBrains-Research/lca-ci-builds-repair)
28
- includes manually curated and assessed 77 data points coming from 32 Python repositories.
29
 
30
- We use Pass@1 metric for CI repair.
31
- Models can be evaluated in three task types:
32
  * `full` – **no** ground truth diffs are used for model evaluation;
33
  * `oracle: files` – ground truth diffs are used to select files that should be corrected to fix the issue;
34
  * `oracle: files, lines` – ground truth diffs are used to select files and code blocks that should be corrected to fix the issue;
 
24
 
25
  "ci_builds_repair": """# CI builds repair\n
26
 
27
+ Our CI builds repair benchmark πŸ€— [JetBrains-Research/lca-ci-builds-repair](https://huggingface.co/datasets/JetBrains-Research/lca-ci-builds-repair)
28
+ includes 77 manually curated and assessed data points coming from 32 Python repositories, which are used to make a model fix a failed build.
29
 
30
+ We use the `Pass@1` metric for CI repair.
31
+ Models can be evaluated in three types of tasks:
32
  * `full` – **no** ground truth diffs are used for model evaluation;
33
  * `oracle: files` – ground truth diffs are used to select files that should be corrected to fix the issue;
34
  * `oracle: files, lines` – ground truth diffs are used to select files and code blocks that should be corrected to fix the issue;