Guides
Larger-Than-Memory Workloads (Out-of-Core Processing)
A key strength of DuckDB is support for larger-than-memory workloads, i.e., it is able to process data sets that are larger than the available system memory (also known as _out-of-core processing_).
It can also run queries where the intermediate results cannot fit into memory.
This section explains the prerequisites, scope, and known limitations of larger-than-memory processing in DuckDB.  
#### Spilling to Disk {#docs:guides:performance:how_to_tune_workloads::spilling-to-disk}  
Larger-than-memory workloads are supported by spilling to disk.
With the default configuration, DuckDB creates the `⟨database_file_name⟩.tmp` temporary directory (in persistent mode) or the `.tmp` directory (in in-memory mode). This directory can be changed using the [`temp_directory` configuration option](#docs:configuration:pragmas::temp-directory-for-spilling-data-to-disk), e.g.:  
```sql
SET temp_directory = '/path/to/temp_dir.tmp/';
```  
#### Blocking Operators {#docs:guides:performance:how_to_tune_workloads::blocking-operators}  
Some operators cannot output a single row until the last row of their input has been seen.
These are called _blocking operators_ as they require their entire input to be buffered,
and are the most memory-intensive operators in relational database systems.
The main blocking operators are the following:  
* _sorting:_ [`ORDER BY`](#docs:sql:query_syntax:orderby)
* _grouping:_ [`GROUP BY`](#docs:sql:query_syntax:groupby)
* _windowing:_ [`OVER ... (PARTITION BY ... ORDER BY ...)`](#docs:sql:functions:window_functions)
* _joining:_ [`JOIN`](#docs:sql:query_syntax:from::joins)  
DuckDB supports larger-than-memory processing for all of these operators.  
#### Limitations {#docs:guides:performance:how_to_tune_workloads::limitations}  
DuckDB strives to always complete workloads even if they are larger-than-memory.
That said, there are some limitations at the moment:  
* If multiple blocking operators appear in the same query, DuckDB may still throw an out-of-memory exception due to the complex interplay of these operators.
* Some [aggregate functions](#docs:sql:functions:aggregates), such as `list()` and `string_agg()`, do not support offloading to disk.
* [Aggregate functions that use sorting](#docs:sql:functions:aggregates::order-by-clause-in-aggregate-functions) are holistic, i.e., they need all inputs before the aggregation can start. As DuckDB cannot yet offload some complex intermediate aggregate states to disk, these functions can cause an out-of-memory exception when run on large data sets.
* The `PIVOT` operation [internally uses the `list()` function](#docs:sql:statements:pivot::internals), therefore it is subject to the same limitation.