Set up Scrapy framework to run on Python 2.7
Question: Is it possible to select which version of Python is used by Scrapy? I am
running Scrapy on Ubuntu 10.04 which ships with Python 2.6. I have Python 2.7
installed on my machine and would like to take advantage of some of the
features of this later version but do not know how to set Scrapy to run on
2.7. When I type "python" into terminal, it runs Python 2.6 ("python2.7" loads
Python 2.7). Ideas?


Answer: The right way to do this is to organize things so that your special Python is
in its own subdirectory that has a bin and lib subdirectory. Then you put that
subdirectory in the `PATH` environment variable before the system binary
directories.

For instance, lets say you have a `/python directory` and you put the python
binary in `/python/bin/python`. Whether you do that by building python from
scratch, copying files, or linking to existing files, is not important. They
will all work. Note that it may not be enough to simply link to the existing
python2.7 binary since that will likely expect to find the Python libraries in
/python/lib if you run it this way.

Second step is to run `export PATH=/python/bin:$PATH`. You can type that at
the shell prompt to experiment, but longer term that should either go in a
`~/.profile` file, or in a special shell script used to run your application,
such as scrapy.

Note that a very popular way for Python developers to do this is to install
and setup virtualenv but if you aren't going to be changing environments every
day, that is probably overkill.

If you have this problem on a lot of machines then you might want to have a
custom build of Python that you can use everywhere such as [the portable
python built with this script](https://github.com/wavetossed/pybuild).

