Data Import
Retrieving Faulty CSV Lines
Being able to read faulty CSV files is important, but for many data cleaning operations, it is also necessary to know exactly which lines are corrupted and what errors the parser discovered on them. For scenarios like these, it is possible to use DuckDB's CSV Rejects Table feature.
By default, this feature creates two temporary tables.  
1. `reject_scans`: Stores information regarding the parameters of the CSV Scanner
2. `reject_errors`: Stores information regarding each CSV faulty line and in which CSV Scanner they happened.  
Note that any of the errors described in our Structural Error section will be stored in the rejects tables. Also, if a line has multiple errors, multiple entries will be stored for the same line, one for each error.  
#### Reject Scans {#docs:data:csv:reading_faulty_csv_files::reject-scans}  
The CSV Reject Scans Table returns the following information:  
<div class="narrow_table"></div>  
| Column name | Description | Type |
|:--|:-----|:-|
| `scan_id` | The internal ID used in DuckDB to represent that scanner | `UBIGINT` |
| `file_id` | A scanner might happen over multiple files, so the file_id represents a unique file in a scanner | `UBIGINT` |
| `file_path` | The file path | `VARCHAR` |
| `delimiter` | The delimiter used e.g., ; | `VARCHAR` |
| `quote` | The quote used e.g., " | `VARCHAR` |
| `escape` | The quote used e.g., " | `VARCHAR` |
| `newline_delimiter` | The newline delimiter used e.g., \r\n | `VARCHAR` |
| `skip_rows` | If any rows were skipped from the top of the file | `UINTEGER` |
| `has_header` | If the file has a header | `BOOLEAN` |
| `columns` | The schema of the file (i.e., all column names and types) | `VARCHAR` |
| `date_format` | The format used for date types | `VARCHAR` |
| `timestamp_format` | The format used for timestamp types| `VARCHAR` |
| `user_arguments` | Any extra scanner parameters manually set by the user | `VARCHAR` |  
#### Reject Errors {#docs:data:csv:reading_faulty_csv_files::reject-errors}  
The CSV Reject Errors Table returns the following information:  
<div class="narrow_table"></div>  
| Column name | Description | Type |
|:--|:-----|:-|
| `scan_id` | The internal ID used in DuckDB to represent that scanner, used to join with reject scans tables | `UBIGINT` |
| `file_id` | The file_id represents a unique file in a scanner, used to join with reject scans tables | `UBIGINT` |
| `line` | Line number, from the CSV File, where the error occurred. | `UBIGINT` |
| `line_byte_position` | Byte Position of the start of the line, where the error occurred. | `UBIGINT` |
| `byte_position` | Byte Position where the error occurred. | `UBIGINT` |
| `column_idx` | If the error happens in a specific column, the index of the column. | `UBIGINT` |
| `column_name` | If the error happens in a specific column, the name of the column. | `VARCHAR` |
| `error_type` | The type of the error that happened. | `ENUM` |
| `csv_line` | The original CSV line. | `VARCHAR` |
| `error_message` | The error message produced by DuckDB. | `VARCHAR` |