jenyag commited on
Commit
159c898
Β·
verified Β·
1 Parent(s): 8f69df6

Update src/tasks_content.py

Browse files
Files changed (1) hide show
  1. src/tasks_content.py +24 -4
src/tasks_content.py CHANGED
@@ -1,12 +1,12 @@
1
  from typing import Optional
2
 
3
  TASKS_PRETTY = {
 
 
 
4
  "commit_message_generation": "Commit Message Generation",
5
  "bug_localization": "Bug Localization on Issue",
6
  "module_summarization": "Module Summarization",
7
- "library_usage": "Library Usage Examples Generation",
8
- "project_code_completion": "Project-level Code Completion",
9
- "bug_localization_build_logs": "Bug Localization on Build Logs",
10
  }
11
  TASKS_PRETTY_REVERSE = {value: key for key, value in TASKS_PRETTY.items()}
12
 
@@ -40,8 +40,28 @@ TASKS_DESCRIPTIONS = {
40
 
41
  For further details on the dataset and the baselines from 🏟️ Long Code Arena Team, refer to `module2text` folder in [our baselines repository](https://github.com/JetBrains-Research/lca-baselines) or to our preprint (TODO).
42
  """,
 
43
  "library_usage": "cool description for Library Usage Examples Generation task",
44
- "project_code_completion": "cool description for Project-level Code Completion task",
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
45
  "bug_localization_build_logs": "cool description for Bug Localization on Build Logs task",
46
  }
47
 
 
1
  from typing import Optional
2
 
3
  TASKS_PRETTY = {
4
+ "library_usage": "Library Usage Examples Generation",
5
+ "bug_localization_build_logs": "Bug Localization on Build Logs",
6
+ "project_code_completion": "Project-level Code Completion",
7
  "commit_message_generation": "Commit Message Generation",
8
  "bug_localization": "Bug Localization on Issue",
9
  "module_summarization": "Module Summarization",
 
 
 
10
  }
11
  TASKS_PRETTY_REVERSE = {value: key for key, value in TASKS_PRETTY.items()}
12
 
 
40
 
41
  For further details on the dataset and the baselines from 🏟️ Long Code Arena Team, refer to `module2text` folder in [our baselines repository](https://github.com/JetBrains-Research/lca-baselines) or to our preprint (TODO).
42
  """,
43
+
44
  "library_usage": "cool description for Library Usage Examples Generation task",
45
+
46
+ "project_code_completion": """# Project-Level Code Completion\n
47
+ Our Project-Level Code Completion πŸ€— [JetBrains-Research/lca-code-completion](https://huggingface.co/datasets/JetBrains-Research/lca-code-completion) includes four datasets:
48
+ - `small-context`: 144 data points,
49
+ - `medium-context: 224 data points,
50
+ - `large-context`: 270 data points,
51
+ - `huge-context`: 296 data points.
52
+
53
+ We use standard Exact Match (EM) metric for one-line code completion.
54
+ We evaluate Exact Match for different line categories:
55
+ - *infile* – functions and classes are from the completion file;
56
+ - *inproject – functions and files are from the repository snapshot;
57
+ - *committed* – functions and classes are from the files that were added on the completion file commit;
58
+ - *common* – functions and classes with common names, e.g., `main`, `get`, etc.;
59
+ - *non-informative* – short/long lines, import/print lines, or comment lines;
60
+ - *random* – lines that doesn't fit to any of previous categories.
61
+
62
+ For further details on the dataset and the baselines from 🏟️ Long Code Arena Team, refer to `code_completion` folder in [our baselines repository](https://github.com/JetBrains-Research/lca-baselines) or to our preprint (TODO).
63
+ """,
64
+
65
  "bug_localization_build_logs": "cool description for Bug Localization on Build Logs task",
66
  }
67