Inter-thread communication with python: Plotting memory-consumption using separate python thread
Question: Withing a python-script I call sequentially different functions (lets say
func_a, func_b, func_c), which process a given set of input data.

The execution takes about 30min.

Within these 30 minutes, I want to track and plot the memory consumption of my
program.

I considered to do this within a separate thread (e.g. my_tracking_thread),
which checks for the current memory usage.

[E.g. like this:](http://fa.bianp.net/blog/2013/different-ways-to-get-memory-
consumption-or-lessons-learned-from-memory_profiler/)

    
    
    import psutil
    process = psutil.Process(os.getpid())
    mem = process.get_memory_info()[0] / float(2 ** 20)
    

Plotting the collected data of my_tracking_thread with matplotlib, I would
like to include as additional information the time-stamps, at which the
different functions started (func_a@4:14, func_b@6:23, func_c@25:48).

Therefore the question: How do I get my_tracking_thread to notify, that
func_{a|b|c} has been started?

Any help appreciated.


Answer: It's difficult to see how you could do that from the other thread. Do you need
to? I'd suggest wrapping them in decorators that log the time they start and
finish, and then when you generate the plot with the memory consumption data
merging in the start/finish information.

    
    
    import collections
    import functools
    import time
    LogEvent = collections.namedtuple('LogEvent', 'function event timestamp')
    
    events = []
    
    def log_start_finish(func):
        @functools.wraps(func)
        def wrapper(*args, **kwargs):
            events.append(LogEvent(func.__name__, 'start', time.time())
            result = func(*args, **kwargs)
            events.append(LogEvent(func.__name__, 'finish', time.time())
            return result
        return wrapper
    
    @log_start_finish
    def func_a(...):
        ....
    
    @log_start_finish
    def func_b(...):
        ....
    
    @log_start_finish
    def func_c(...):
        ....
    

