Memory efficient way to load large entity set for data export I'm working on an Excel exporting feature in a Drupal 8 site.

Obviously, I can load all entites of certain type like this:

$user = \Drupal::currentUser();

$entity_query = \Drupal::service('entity.query')-&gt;get('some_type');
$entity_query-&gt;sort('created', 'DESC');

$ids = $entity_query-&gt;execute();
$entites = $this-&gt;storage-&gt;loadMultiple($ids);

foreach ($entities as $entity) {
    // build some Excel row with the entity information
    // ...
}


But if you're doing this on a large entity set, you'll be running out of memory very soon. I think the reason why Drupal would run out of memory is the extensive caching involves in the entity query and loading process.

Is there a more memory-efficient way to load an indefinite size of entity set without running out of memory?

(Note: Let's pretend the Excel library is very clever and do not take up memory. That's another issue for another day.)
Batches allow heavy processing to be spread out over several page
  requests, ensuring that the processing does not get interrupted
  because of a PHP timeout, while allowing the user to receive feedback
  on the progress of the ongoing operations. It also can reduce out of
  memory situations.
  
  Source: Example: Batch API 


You need to use the Batch API. There is no official docs for D8, but you can look at the batch_example module and the CSV Import module.

Good conceptual read, Drupal 8 Batch Processing but a bad code example.