python load zip with modules from memory
Question: So lets say I have a zip file with modules/classes inside. I then read this
file - read binary ("rb") to store it into memory. How would I take this zip
file in memory and load a module from it. Would I need to write an import hook
for this? One cannot simply run exec on binary zip data from memory, can they?

I know its simple just to load a module from a plain zip file on disk as this
is done automatically by python2.7. I , however; want to know if this is
possible via memory.

Update: A lot of people are mentioning about importing the zip from disk. The
issue is specifically that I want to import the zip from memory NOT disk. I
obviously will read it from disk and into memory byte by byte. I want to take
all these bytes from within memory that make up the zip file and use it as a
regular import.


Answer: **EDIT:** Fixed the ZipImporter to work for everything (I think)

Test Data:

    
    
    mkdir mypkg
    vim mypkg/__init__.py
    vim mypkg/test_submodule.py
    

`__init__.py` Contents:

    
    
    def test():
        print("Test")
    

`test_submodule.py` Contents:

    
    
    def test_submodule_func():
        print("This is a function")
    

Create Test Zip (on mac):

    
    
    zip -r mypkg.zip mypkg
    rm -r mypkg # don't want to accidentally load the directory
    

Special zip import in `inmem_zip_importer.py`:

    
    
    import os
    import imp
    import zipfile
    
    class ZipImporter(object):
        def __init__(self, zip_file):
            self.z = zip_file
            self.zfile = zipfile.ZipFile(self.z)
            self._paths = [x.filename for x in self.zfile.filelist]
    
        def _mod_to_paths(self, fullname):
            # get the python module name
            py_filename = fullname.replace(".", os.sep) + ".py"
            # get the filename if it is a package/subpackage
            py_package = fullname.replace(".", os.sep, fullname.count(".") - 1) + "/__init__.py"
            if py_filename in self._paths:
                return py_filename
            elif py_package in self._paths:
                return py_package
            else:
                return None
    
        def find_module(self, fullname, path):
            if self._mod_to_paths(fullname) is not None:
                return self
            return None
    
        def load_module(self, fullname):
            filename = self._mod_to_paths(fullname)
            if not filename in self._paths:
                raise ImportError(fullname)
            new_module = imp.new_module(fullname)
            exec self.zfile.open(filename, 'r').read() in new_module.__dict__
            new_module.__file__ = filename
            new_module.__loader__ = self
            if filename.endswith("__init__.py"):
                new_module.__path__ = [] 
                new_module.__package__ = fullname
            else:
                new_module.__package__ = fullname.rpartition('.')[0]
            return new_module
    

Use:

    
    
    In [1]: from inmem_zip_importer import ZipImporter
    In [2]: sys.meta_path.append(ZipImporter(open("mypkg.zip", "rb")))
    In [3]: from mypkg import test
    In [4]: test()
    Test function
    In [5]: from mypkg.test_submodule import test_submodule_func
    In [6]: test_submodule_func()
    This is a function
    

* * *

(from efel) one more thing... :

To read directly from memory one would need to do this :

    
    
    f = open("mypkg.zip", "rb")
    
    # read binary data we are now in memory
    data = f.read()
    
    f.close()  #important! close the file! we are now in memory
    
    # at this point we can essentially delete the actual on disk zip file
    
    # convert in memory bytes to file like object
    zipbytes = io.BytesIO(data)
    
    zipfile.ZipFile(zipbytes)
    

