Drupal queue processing at cron run I use ultimate cron to fill in a queue with some nids.

To add the items in the queue, I use something like:

$queue = DrupalQueue::get('my_queue');
foreach($urls as $nid=&gt;$url){
  sleep(1);
  $response = drupal_http_request($url,array('timeout'=&gt;3,'method'=&gt;'head'));
  if($response-&gt;code=='404'){
    $queue-&gt;createItem(array('nid' =&gt; $nid));
  }


This works OK.

When the time comes, another cron job is executed that deletes those nodes. This is done like:

$queue = DrupalQueue::get('my_queue');
for($i=0; $i&lt;$queue-&gt;numberOfItems(); $i++){
  $item = $queue-&gt;claimitem();
  node_delete($item-&gt;data['nid']);
  $queue-&gt;deleteItem($item);
}


This is not working OK; there is a small issue: Everytime the second job runs, only half of the items in the queue are processed (deleted). If there are 3000 items in the queue, only 1500 are deleted. If there are 1500 items, only 750 are deleted and so on.

Why is that? Is there a way to control it?
So you have 2 queues walking all over eachother. The timeout value is how long the item is Leased to have work done upon it. The lease-time does not guarantee your processing will finish it time. A Leased item will not be able to be deleted until its lease has expired.

So you should analyze the tasks being run and adjust your timeout to be larger than the longest processing time you have recorded as occurring. I typically do a 2x or 10x as much Lease-time as my longest evaluated processing time. Once an item has completed processing its Lease is expired (so over guessing doesn't really hurt you here).

Alternatively, instead of having 2 queues you could do all this with 1 queue. To try to get around contention. You could insert into a single queue all your NIDs with an additional value of task; and at the end of that queue append a new list of NIDs with an additional flag value of delete or purge. Adjust your worker function to examine a NID and the action to perform on the NID; task or delete.

For example your createItem could look like:

// Add all NIDs for task to be performed.
$queue-&gt;createItem(array('nid' =&gt; $nid, 'action' =&gt; 'do-task'));
// in the same list later on queue the same NIDs to be deleted.
// by setting a huuuuge lease time you will never accidently purge an
// uncompleted task.
$queue-&gt;createItem(array('nid' =&gt; $nid, 'action' =&gt; 'do-delete'));


You may find working with a single queue -- easier than juggling 2 queues.