uparekh01151 commited on
Commit
3cf16fb
Β·
1 Parent(s): 5cc5417

docs: update README to focus on data engineering tools

Browse files

- Update description to emphasize systematic benchmarking for data engineering
- Remove Go-related content (web services, concurrency, HTTP handlers)
- Focus on Python data processing, ETL pipelines, data transformation
- Add documentation generation for technical docs and API documentation
- Align project structure with data engineering focus

Files changed (1) hide show
  1. README.md +11 -8
README.md CHANGED
@@ -19,12 +19,12 @@ tags:
19
 
20
  # DataEngEval
21
 
22
- A comprehensive evaluation platform for AI models across SQL generation and code generation. Compare model performance with standardized metrics on real-world datasets including NYC Taxi queries, Python algorithms, and Go web services.
23
 
24
  ## πŸš€ Features
25
 
26
- - **Multi-use-case evaluation**: SQL generation, Python code, Go services
27
- - **Real-world datasets**: NYC Taxi, sorting algorithms, HTTP handlers, concurrency patterns
28
  - **Comprehensive metrics**: Correctness, execution success, syntax validation, performance
29
  - **Remote inference**: Uses Hugging Face Inference API (no local model downloads)
30
  - **Mock mode**: Works without API keys for demos
@@ -37,9 +37,12 @@ A comprehensive evaluation platform for AI models across SQL generation and code
37
  - **Metrics**: Correctness, execution, result matching, dialect compliance
38
 
39
  ### Code Generation
40
- - **Python**: Algorithms, data structures, object-oriented programming
41
- - **Go**: Web services, concurrency, HTTP handlers
42
- - **Metrics**: Syntax correctness, compilation success, execution success, code quality
 
 
 
43
 
44
  ## πŸ—οΈ Project Structure
45
 
@@ -59,8 +62,8 @@ dataeng-leaderboard/
59
  β”‚ └── utils/ # Utility functions
60
  β”œβ”€β”€ tasks/ # Multi-use-case datasets
61
  β”‚ β”œβ”€β”€ sql_generation/ # SQL generation tasks
62
- β”‚ β”œβ”€β”€ code_generation/ # Code generation tasks
63
- β”‚ └── documentation/ # Documentation tasks
64
  β”œβ”€β”€ prompts/ # SQL generation templates
65
  └── test/ # Test files
66
  ```
 
19
 
20
  # DataEngEval
21
 
22
+ A comprehensive evaluation platform for systematically benchmarking performance across various models and programming languages, focusing on data engineering tools and technologies.
23
 
24
  ## πŸš€ Features
25
 
26
+ - **Multi-use-case evaluation**: SQL generation, Python data processing, documentation generation
27
+ - **Real-world datasets**: NYC Taxi queries, data transformation algorithms, technical documentation
28
  - **Comprehensive metrics**: Correctness, execution success, syntax validation, performance
29
  - **Remote inference**: Uses Hugging Face Inference API (no local model downloads)
30
  - **Mock mode**: Works without API keys for demos
 
37
  - **Metrics**: Correctness, execution, result matching, dialect compliance
38
 
39
  ### Code Generation
40
+ - **Python**: Data processing algorithms, ETL pipelines, data transformation functions
41
+ - **Metrics**: Syntax correctness, execution success, data processing accuracy, code quality
42
+
43
+ ### Documentation Generation
44
+ - **Technical Documentation**: API documentation, system architecture, data pipeline documentation
45
+ - **Metrics**: Content accuracy, completeness, technical clarity, formatting quality
46
 
47
  ## πŸ—οΈ Project Structure
48
 
 
62
  β”‚ └── utils/ # Utility functions
63
  β”œβ”€β”€ tasks/ # Multi-use-case datasets
64
  β”‚ β”œβ”€β”€ sql_generation/ # SQL generation tasks
65
+ β”‚ β”œβ”€β”€ code_generation/ # Python data processing tasks
66
+ β”‚ └── documentation/ # Technical documentation tasks
67
  β”œβ”€β”€ prompts/ # SQL generation templates
68
  └── test/ # Test files
69
  ```