Python include Scrapy from subdirectory
Question: I would like to know if there's a way that I can put Scrapy into a
subdirectory and import it. I did this with BeautifulSoup, rather than
installing it, I just drop the bs4 directory into the directory of my app, and
import it:

`from bs4 import BeautifulSoup`

In the source that I downloaded from scrapy.org there is no scrapy.py so I
tried importing

`from scrapy import *`

This returned a bunch of errors.

    
    
    Traceback (most recent call last):
    File "C:\Users\Kat\Desktop\linkscrape\cookie.py", line 1, in <module> from scrapy import *
    File "C:\Users\Kat\Desktop\linkscrape\scrapy\__init__.py", line 27, in <module>
    from . import _monkeypatches
    File "C:\Users\Kat\Desktop\linkscrape\scrapy\_monkeypatches.py", line 2, in <module>
    from six.moves import copyreg
    ImportError: No module named six.moves
    

Is there any way that I can simply include this to make it easy to migrate the
app from computer to computer, or does this have to be installed? Thanks.


Answer: Don't do that. You will hurt yourself.

To achieve easy portability, use [virtualenv](http://docs.python-
guide.org/en/latest/dev/virtualenvs/) \- it's a golden standard of Python
development.

Create a file named
[requirements.txt](https://pip.readthedocs.org/en/1.1/requirements.html) in a
root directory of your project, and write all necessary dependencies, like
this:

    
    
    python-dateutil==2.4.2
    Scrapy==1.0.4
    XlsxWriter==0.7.7
    

When you are setting up the environment, create a fresh virtualenv and then
simply:

    
    
    pip install -r requirements.txt
    

Voilà! You have a working environment.

In case of Scrapy, it's even more important because in production Scrapy is
typically deployed using
[scrapyd](https://scrapyd.readthedocs.org/en/latest/), where having a correct
[package](https://python-packaging.readthedocs.org/en/latest/) with a version
number and fixed requirements is absolutely necessary.

