Long-running cron task never fully executed
After a long battle analyzing how to execute a long-running task as a Drupal Cron, finally I found a generic solution that involves the Queue API and Cron. Maybe this can help other people who is struggling with a similar problem. Here are the steps I followed:


Think about how to execute your long-running task as chunks in terms of Drupal Queue items. You need to represent all tasks as queue items. For example, the first item (or action) may be to initialize the complex work that needs to be done. Then, the following items may execute that work by pieces. Each item (or chunk) has to be processed individually, so no PHP Timeout or PHP Memory Exhausted error appear.
Create a new Cron that will create the queue and its items. Make sure that you also implement the hook_cron_queue_info().
Implement the queue item process function you defined in the above hook. If you need to pass information between each item process, you can use Drupal's variable_get(...) and variable_set(...). If you need a more complex structure, you can define a database with Schema API in the module my_module.install file.
Now create a new function that will read and process one or more items of your queue (if not empty).


For example:

function my_module_process_my_queue() {
  $my_queue = DrupalQueue::get('my_module_queue');
  if ($my_queue-&gt;numberOfItems() &gt; 0) {
    $action = $my_queue-&gt;claimItem();
    if ($action) {
      my_module_queue_process_item($action-&gt;data);
      $my_queue-&gt;deleteItem($action);
    }
  }
}



Install a Cron module (such as Elysia Cron) or add a cron job in your server configuration. At least, you need to specify the execution of 2 Cron functions. The first one (from step 2) will run the moment you prefer. In my case, it has to run once a day. The second one will be a simple Cron that will call the function from step 4. That second Cron will run every X minutes. Where X means the maximum time execution that a queue item processing may take. If you do not install a Cron module, you do not need the second Cron function. In my case, I decided to use Linux crontab configuration and Drush.


For example:

*/2 * * * * cd /path/to/your/drupal/installation &amp;&amp; export DRUSH_PHP=/path/to/your/bin/php &amp;&amp; drush ev "my_module_process_my_queue();"



Finally, execute your crons or wait until they are executed. Once a day, the queue will be filled with new queue items. Those items will be processed during the first Cron execution and every X minutes with the other one. So, after some time, all queue items will be processed and the long-running task will finish.