improving speed of Python module import
Question: The question of how to speed up importing of Python modules has been asked
previously ([Speeding up the python "import"
loader](http://stackoverflow.com/questions/2010255/speeding-up-the-python-
import-loader) and [Python -- Speed Up
Imports?](http://stackoverflow.com/questions/6025635/python-speed-up-imports))
but without specific examples and has not yielded accepted solutions. I will
therefore take up the issue again here, but this time with a specific example.

I have a Python script that loads a 3-D image stack from disk, smooths it, and
displays it as a movie. I call this script from the system command prompt when
I want to quickly view my data. I'm OK with the 700 ms it takes to smooth the
data as this is comparable to MATLAB. However, it takes an additional 650 ms
to import the modules. So from the user's perspective the Python code runs at
half the speed.

This is the series of modules I'm importing:

    
    
    import numpy as np
    import matplotlib.pyplot as plt
    import matplotlib.animation as animation
    import scipy.ndimage
    import scipy.signal
    import sys
    import os
    

Of course, not all modules are equally slow to import. The chief culprits are:

    
    
    matplotlib.pyplot   [300ms]
    numpy               [110ms]
    scipy.signal        [200ms]
    

I have experimented with using `from`, but this isn't any faster. Since
Matplotlib is the main culprit and it's got a reputation for slow screen
updates, I looked for alternatives. One is PyQtGraph, but that takes 550 ms to
import.

I am aware of one obvious solution, which is to call my function from an
interactive Python session rather than the system command prompt. This is fine
but it's too MATLAB-like, I'd prefer the elegance of having my function
available from the system prompt.

I'm new to Python and I'm not sure how to proceed at this point. Since I'm
new, I'd appreciate links on how to implement proposed solutions. Ideally, I'm
looking for a simple solution (aren't we all!) because the code needs to be
portable between multiple Mac and Linux machines.


Answer: you could build a simple server/client, the server running continuously making
and updating the plot, and the client just communicating the next file to
process.

I wrote a simple server/client example based on the basic example from the
`socket` module docs: <http://docs.python.org/2/library/socket.html#example>

here is server.py:

    
    
    # expensive imports
    import numpy as np
    import matplotlib.pyplot as plt
    import matplotlib.animation as animation
    import scipy.ndimage
    import scipy.signal
    import sys
    import os
    
    # Echo server program
    import socket
    
    HOST = ''                 # Symbolic name meaning all available interfaces
    PORT = 50007              # Arbitrary non-privileged port
    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    s.bind((HOST, PORT))
    s.listen(1)
    while 1:
        conn, addr = s.accept()
        print 'Connected by', addr
        data = conn.recv(1024)
        if not data: break
        conn.sendall("PLOTTING:" + data)
        # update plot
        conn.close()
    

and client.py:

    
    
    # Echo client program
    import socket
    import sys
    
    HOST = ''    # The remote host
    PORT = 50007              # The same port as used by the server
    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    s.connect((HOST, PORT))
    s.sendall(sys.argv[1])
    data = s.recv(1024)
    s.close()
    print 'Received', repr(data)
    

you just run the server:

    
    
    python server.py
    

which does the imports, then the client just sends via the socket the filename
of the new file to plot:

    
    
    python client.py mytextfile.txt
    

then the server updates the plot.

On my machine running your imports take 0.6 seconds, while running `client.py`
0.03 seconds.

