python : print output of each thread to seperate file no processes
Question: I have several threads and each thread writes output to stdout. However I want
to redirect the ouput of each thread to a separate file independently of each
other, then merge them to keep he flow of each thread together.

What I mean is the following:

Thread1 writes every print, every exception and every other ouput into
file1.log Thread2 writes every print, every exception and every other ouput
into file2.log and so on. So what I'm looking for is to set the stdout for
each thread exclusivly. However setting the stdout only works globally mean
that Thread1 and Tread2 will always write to the same defined stdout. I have
not found out yet how to do this. I can't use processes though, because f
another issue.

How can I do that?


Answer: The Python [**`logging`**](http://docs.python.org/2/library/logging.html)
module is [thread-safe](http://docs.python.org/2/library/logging.html#thread-
safety).

Use it to create an individual logger for each thread, and register a
[`FileHandler`](http://docs.python.org/2/library/logging.handlers.html#logging.FileHandler)
to (also) log to a file:

    
    
    logger = logging.getLogger('thread-1')
    file_handler = logging.FileHandler('thread-1.log')
    logger.addHandler(file_handler)
    

Here's a more complete example:

    
    
    import logging
    import random
    import threading
    import time
    
    NUM_THREADS = 5
    
    
    def worker(delay, logger):
        """A toy worker function, taking the logger as an argument.
        """
        logger.info("Starting work...")
        for i in range(3):
            logger.info('Sleeping %0.02f', delay)
            time.sleep(delay)
        logger.info('Done.')
    
    
    for n in range(1, NUM_THREADS + 1):
        # create the thread's logger
        logger = logging.getLogger('thread-%s' % n)
        logger.setLevel(logging.DEBUG)
    
        # create a file handler writing to a file named after the thread
        file_handler = logging.FileHandler('thread-%s.log' % n)
    
        # create a custom formatter and register it for the file handler
        formatter = logging.Formatter('(%(threadName)-10s) %(message)s')
        file_handler.setFormatter(formatter)
    
        # register the file handler for the thread-specific logger
        logger.addHandler(file_handler)
    
        delay = random.random()
        t = threading.Thread(target=worker, args=(delay, logger))
        t.start()
    
    
    main_thread = threading.currentThread()
    for t in threading.enumerate():
        if t is not main_thread:
            t.join()
    

This will give you five logfiles, `thread-1.log` through `thread-5.log`,
containing only the output of the respective thread:

**`thread-1.log`**

    
    
    (Thread-1  ) Starting work...
    (Thread-1  ) Sleeping 0.53
    (Thread-1  ) Sleeping 0.53
    (Thread-1  ) Sleeping 0.53
    (Thread-1  ) Done.
    

* * *

If you still want to log to the console, simply create a
[`StreamHandler`](http://docs.python.org/2/library/logging.handlers.html#streamhandler)
and attach it to your logger:

    
    
    stream_handler = logging.StreamHandler()
    logger.addHandler(stream_handler)
    

This will log to `STDERR` by default. If you want `STDOUT`, use

    
    
    logging.StreamHandler(sys.stdout)
    

For more information on using the Python `logging` module, see the [Advanced
Logging Tutorial](http://docs.python.org/2/howto/logging.html#logging-
advanced-tutorial).

