Can I force cron to process all of my queued tasks? My D7 module uses hook_cron to mirror a big external datasource locally. For each new or updated row, $queue->createItem() schedules a task requiring several external API round trips to build the locally mirrored row.

I might queue 750 tasks but Drupal only runs 50 on a cron. How do I make all queued tasks run on each cron?

Please note:


There are no errors reported in /var/log/apache2/error.log 
The latest message in admin/reports/dblog reports "Cron run completed." (looks successful)
Basically PHP has memory and execution time limitations, so if you try to do a huge import of an external database you'll run out of runtime if not memory.

So the solution is to spread it out over many http requests, each doing a smaller chunk of the job.

For this, Drupal has the Batch API, which you can call from cron as needed.

Here's the Examples project's Batch example (with tests!) to guide the way. And a decent explanation with some code samples.