Does Python's main thread get garbage collected when it stops?
Question: In a multi-threaded Python process I have a number of _non-daemon_ threads, by
which I mean threads which keep the main process alive even after the main
thread has exited / stopped.

My non-daemon threads hold [weak
references](http://docs.python.org/release/2.6.5/library/weakref.html#weakref.ref)
to certain objects in the main thread, but when the main thread ends (control
falls off the bottom of the file) these objects do _not_ appear to be garbage
collected, and my weak reference finaliser callbacks don't fire.

Am I wrong to expect the main thread to be garbage collected? I would have
expected that the thread-locals would be deallocated (i.e. garbage
collected)...

What have I missed?

* * *

**Supporting materials**

Output from `pprint.pprint( threading.enumerate() )` showing the main thread
has stopped while others soldier on.

    
    
    [<_MainThread(MainThread, stopped 139664516818688)>,
     <LDQServer(testLogIOWorkerThread, started 139664479889152)>,
     <_Timer(Thread-18, started 139663928870656)>,
     <LDQServer(debugLogIOWorkerThread, started 139664437925632)>,
     <_Timer(Thread-17, started 139664463103744)>,
     <_Timer(Thread-19, started 139663937263360)>,
     <LDQServer(testLogIOWorkerThread, started 139664471496448)>,
     <LDQServer(debugLogIOWorkerThread, started 139664446318336)>]
    

And since someone always asks about the use-case...

My network service occasionally misses its real-time deadlines (which causes a
total system failure in the worst case). This turned out to be because logging
of (important) DEBUG data would block whenever the file-system has a tantrum.
So I am attempting to retrofit a number of established specialised logging
libraries to defer blocking I/O to a worker thread.

Sadly the established usage pattern is a mix of short-lived logging channels
which log overlapping parallel transactions, and long-lived module-scope
channels which are never explicitly closed.

So I created a decorator which defers method calls to a worker thread. The
worker thread is non-daemon to ensure that all (slow) blocking I/O completes
before the interpreter exits, and holds a weak reference to the client-side
(where method calls get enqueued). When the client-side is garbage collected
the weak reference's callback fires and the worker thread knows no more work
will be enqueued, and so will exit at its next convenience.

This seems to work fine in all but one important use-case: when the logging
channel is in the main thread. When the main thread stops / exits the logging
channel is _not_ finalised, and so my (non-daemon) worker thread lives on
keeping the entire process alive.


Answer: It's a bad idea for your main thread to end without calling `join` on all non-
daemon threads, or to make any assumptions about what happens if you don't.

* * *

If you don't do anything very unusual, CPython (at least
[2.0](http://hg.python.org/cpython/file/2.0/Lib/threading.py#l457)-[3.3](http://hg.python.org/cpython/file/3.3/Lib/threading.py#l829))
will cover for you by automatically calling `join` on all non-daemon threads
as pair of `_MainThread._exitfunc`. This isn't actually documented, so you
shouldn't rely on it, but it's what's happening to you.

Your main thread hasn't actually exited at all; it's blocking inside its
`_MainThread._exitfunc` trying to `join` some arbitrary non-daemon thread. Its
objects won't be finalized until the `atexit` handler is called, which doesn't
happen until after it finishes joining all non-daemon threads.

* * *

Meanwhile, if you avoid this (e.g., by using `thread`/`_thread` directly, or
by detaching the main thread from its object or forcing it into a normal
`Thread` instance), what happens? It isn't defined. The `threading` module
makes no reference to it at all, but in CPython 2.0-3.3, and likely in any
other reasonable implementation, it falls to the [`thread`/`_thread`
module](http://docs.python.org/2/library/thread.html#module-thread) to decide.
And, as the docs say:

> When the main thread exits, it is system defined whether the other threads
> survive. On SGI IRIX using the native thread implementation, they survive.
> On most other systems, they are killed without executing try ... finally
> clauses or executing object destructors.

So, if you manage to avoid `join`ing all of your non-daemon threads, you have
to write code that can handle both having them hard-killed like daemon
threads, and having them continue running until exit.

If they do continue running, at least in CPython 2.7 and 3.3 on POSIX systems,
that the main thread's OS-level thread handle, and various higher-level Python
objects representing it, may be still retained, and not get cleaned up by the
GC.

* * *

On top of that, even if everything were released, you can't rely on the GC
ever deleting anything. If your code depends on deterministic GC, there are
many cases you can get away with it in CPython (although your code will then
break in PyPy, Jython, IronPython, etc.), but at exit time is not one of them.
CPython can, and will, leak objects at exit time and let the OS sort 'em out.
(This is why writable files that you never close may lose the last few
writes—the `__del__` method never gets called, and therefore there's nobody to
tell them to `flush`, and at least on POSIX the underlying `FILE*` doesn't
automatically flush either.)

If you want something to be cleaned up when the main thread finishes, you have
to use some kind of `close` function rather than relying on `__del__`, and you
have to make sure it gets triggered via a `with` block around the main block
of code, an `atexit` function, or some other mechanism.

* * *

One last thing:

> I would have expected that the thread-locals would be deallocated (i.e.
> garbage collected)...

Do you actually have thread locals somewhere? Or do you just mean locals
and/or globals that are only accessed in one thread?

