Scrapy on Mac OS 10.10 tries to use Python 3.4
Question: I've had some trouble installing Scrapy on Yosemite. I've gotten all the
dependencies installed, but when I try to run scrapy I get this error.

    
    
    $ scrapy crawl my_crawler
    ----------
    File "/Library/Frameworks/Python.framework/Versions/3.4/bin/scrapy", line 5, in <module>
         from pkg_resources import load_entry_point
    File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/pkg_resources.py", line 2697, in <module>
    working_set.require(__requires__)
    File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/pkg_resources.py", line 669, in require
    needed = self.resolve(parse_requirements(requirements))
    File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/pkg_resources.py", line 572, in resolve
    raise DistributionNotFound(req)
    pkg_resources.DistributionNotFound: cryptography>=0.2.1
    

I know that scrapy doesn't support 3.4, but for some reason its trying to use
it. Anyone got any suggestions?


Answer: **I solved this yesterday on my server, but the solution was found using
Google Translate on Chinese websites**

    
    
    pkg_resources.DistributionNotFound: cryptography>=0.2.1
    

This problem indicates that you are missing a package on your distribution
called libffi-dev.

Please locate that package for OSX and install it. Use the easy_install
utility to install the cryptography 0.2.2 package for Python.

You're scrapy application should work after you completely meet the
dependencies.

Python is masking the true dependency issue by reporting it as a problem with
the cryptography packages, when in fact it is a missing [libffi-
dev](https://trac.macports.org/browser/trunk/dports/devel/libffi/Portfile)
package for OSX. You may explicitly need the DEV package for libffi.

Good luck! I was able to solve this by meeting the libffi dependency.

