How to make Python's multiprocessing Queue's .empty() method return the correct value? Or alternatives?
Question: I have this snippet that uses the `Queue` class from the `multiprocess`
module. I am very confused that the `.empty()` method of an instance of
`Queue` does not give me a correct value as i would expect. This is my code:

    
    
    from time import sleep
    from multiprocessing import Queue, Lock
    
    foo = Queue()
    locker = Lock()
    
    with locker:  # even with this, still True
        foo.put("bar")
    
    print(foo.empty())  # True, obviously not
    print(foo.empty())  # True
    print(foo.empty())  # True
    print(foo.qsize())  # 1L
    print(foo.empty())  # True
    

However, if i use the `sleep` function from `time`, as in cause a
chronological delay in the execution. It works.

    
    
    from time import sleep
    from multiprocessing import Queue, Lock
    
    foo = Queue()
    locker = Lock()
    
    foo.put("bar")
    
    sleep(0.01)
    
    print(foo.empty())  # False
    print(foo.empty())  # False
    print(foo.empty())  # False
    print(foo.qsize())  # 1L
    print(foo.empty())  # False
    

I know my alternative is the `.qsize() > 0` expression, but i am sure that i
just doing this in a wrong way.

What am i doing wrong?

***EDIT***

I understand now that is it _unreliable_ , thank you @Mathias Ettinger. Any
clean alternatives? I need to know hot to reliably tell if my `Queue` is empty
or not.


Answer: Unfortunately, the Queue's complex implementation that means that `.empty()`
isn't guaranteed to be correct immediately, even though `.qsize()` is.

Since `.qsize()` _is_ supported on your platform (which is [not true
everywhere](https://docs.python.org/2/library/multiprocessing.html#multiprocessing.Queue.qsize)),
you can re-implement the `.empty()` check in terms of `.qsize()`, and this
will work for you:

    
    
    # mp.Queue() is a function, not a class, so we need to find the true class
    # to subclass
    import multiprocessing.queues
    
    class XQueue(multiprocessing.queues.Queue):
        def empty(self):
            try:
                return self.qsize() == 0
            except NotImplementedError:  # OS X -- see qsize() implementation
                return super(XQueue, self).empty()
    

Under the hood, the Queue `.put()`s your objects to a buffer and increments an
interprocess semaphore, while a hidden daemon thread is responsible for
draining the buffer and serializing its contents to a pipe. (Consumers then
`.get()` by reading from this pipe.) So, that's why sleeping in your example
works: the daemon thread has enough time to move the object from in-memory
buffer to I/O representation before you call `.empty()`.

As an aside, I find this behavior _very_ surprising. Normally when we say a
queue's size methods (empty/full/count) are "unreliable," we mean they're
_accurate and consistent_ , but _possibly immediately stale_ because another
producer or consumer might have changed the queue. (Most single-process
multithreaded queues, like python's `Queue.Queue`, for example, are
"unreliable" but consistent in this sense.) In this case, the default
`.empty()` method really can be inconsistent with the state of the queue.

