import local python module in HTCondor
Question: This concerns the importing of my own python modules in a HTCondor job.

Suppose 'mymodule.py' is the module I want to import, and is saved in
directory called a XDIR. In another directory called YDIR, I have written a
file called xImport.py:

    
    
    #!/usr/bin/env python
    import os
    import sys
    print sys.path
    import numpy
    import mymodule
    

and a condor submit file:

    
    
    executable = xImport.py
    getenv = True
    universe = Vanilla
    output = xImport.out
    error = xImport.error
    log = xImport.log
    queue 1
    

The result of submitting this is that, in xImport.out, the sys.path is printed
out, showing XDIR. But in xImport.error, there is an ImporError saying 'No
module named mymodule'. So it seems that the path to mymodule is in sys.path,
but python does not find it. I'd also like to mention that error message says
that the ImportError originates from the file

    
    
    /mnt/novowhatsit/YDIR/xImport.py
    

and not `YDIR/xImport.py`.

How can I edit the above files to import mymodule.py?


Answer: When condor runs your process, it creates a directory on that machine (usually
on a local hard drive). It sets that as the working directory. That's probably
the issue you are seeing. If XDIR is local to the machine where you are
running condor_submit, then it's contents don't exist on the remote machine
where the xImport.py is running.

Try using the .submit feature transfer_input_files mechanism (see
<http://research.cs.wisc.edu/htcondor/manual/v7.6/2_5Submitting_Job.html>) to
copy the mymodule.py to the remote machines.

