Python: Importing from a running instance, or an alternative
Question: I have 2 files, data.py and interpret.py.

data.py:

    
    
    X = cPickle.load(open("X","r"))
    

interpret.py:

    
    
    from data import X
    query = raw_input("Enter query")
    
    #do something with query and X
    

Object X which is very huge. I dumped it using cPickle. interpret.py will be
invoked by another program and because of the way import currently works,
everytime interpret.py is executed, it loads data.py which loads X, and since
X is very big it takes a lot of time.

I want to be able to do something like this.

data.py:

    
    
    from time import sleep
    X = cPickle.load(open("X","r"))
    sleep(10**10) #Sleep for eternity
    

and then run data.py

interpret.py:

    
    
    from data import X #import from live instance of data.py
                       #don't load X all over again
    query = raw_input("Enter query")
    
    #do something with query and X
    

How can I do this? If I cannot import from a live instance, is there an
alternative to do what I want to do. I am not communicating very well, please
tell me which parts you don't understand and I will try to clarify further.


Answer: From what I can tell you want to be running a daemon. For instance instead of
importing data from interpret, data would run as a standalone daemon which
interpret would communicate with over something like a socket.

But at this point you've basically written a database, so why not just
translate whatever data you have into some database and just communicate with
a database?

Edit: The problem is that you want the huge object to persist between
processes, but it can't work like that persay. Every time you call intepret, a
new process is spawned which means that there is no access to the object
generated by the previous process. That's why daemons (and databases) exist.

