Python 2.7, Ubuntu 14.04: directory to load data from on sys.path, but file still not found
Question: Ok, this is a little bit of a strange one. I had some Python code that needed
to load data from another folder using scipy's loadmat (this bit is not really
relevant, but explains the remainder of the code). Before, I added the path at
the beginning of the file via

    
    
    path_to_add = os.path.abspath(os.path.join(os.path.dirname(__file__), '../data/unit_tests'))
    if not path_to_add in sys.path:
        sys.path.insert(1, path_to_add)
    

and would load the file via

    
    
    mat_file = loadmat('test_kernel',squeeze_me=False)
    

This code ran fine in Ubuntu 12.04. However, when I try running it in 14.04,
it gives the error:

    
    
    IOError: [Errno 2] No such file or directory: 'test_kernel.mat'
    

I printed out sys.path using print sys.path and sure enough, the right
absolute directory with '/data/unit_tests' is present. If I go over to this
folder and try to load the file using loadmat, it loads fine. But for whatever
reason, even though the path is present on sys.path, I still get that error
when I execute the script in another folder. Anyone have any ideas why?


Answer: What version of scipy were you using and what version are you using now? You
can check by running:

    
    
    import scipy
    print scipy.__version__
    

As far as I can tell in my version of scipy (0.14.0) it doesn't use the
sys.path at all to find files when running `loadmat`. I've checked the source
code and it looks like it simply passes in the filename to the builtin `open`
function. Is there a chance on the Ubuntu 12.04 system you were running the
script from the same directory as the mat file? As a sanity check try running
the `loadmat` function with the full path to the mat file.

**Edit**

Here's a question with the same general problem: [open() can't find file given
path relative to PYTHONPATH](http://stackoverflow.com/questions/12391709/open-
cant-find-file-given-path-relative-to-pythonpath)

The usual way to do what you are doing is either have a global "data"
directory so when you need to load a data file you prefix the filename with
that directory. Another way is to specify things as absolute/relative paths to
the python script. If you get a little fancier and create a python package
there are ways to load data files relative to the python package itself.

