scrapy tutorial: cannot run scrapy crawl dmoz
Question: I'm asking a new question because I'm aware I wasn't clear enough in the last
one. I'm trying to follow the scrapy tutorial, but I'm stuck in the crucial
step, the "scrapy crawl dmoz' command. The code is this one (I have written
that in the python shell and save it typing .py extension):

    
    
    ActivePython 2.7.2.5 (ActiveState Software Inc.) based on
    Python 2.7.2 (default, Jun 24 2011, 12:20:15) 
    [GCC 4.2.1 (Apple Inc. build 5664)] on darwin
    Type "copyright", "credits" or "license()" for more information.
    >>> from scrapy.spider import BaseSpider
    
    class dmoz(BaseSpider):
        name = "dmoz"
        allowed_domains = ["dmoz.org"]
        start_urls = [
            "http://www.dmoz.org/Computers/Programming/Languages/Python/Books/",
            "http://www.dmoz.org/Computers/Programming/Languages/Python/Resources/"
        ]
    
        def parse(self, response):
            filename = response.url.split("/")[-2]
            open(filename, 'wb').write(response.body)
    
    >>> 
    

The directory I'm using should be fine, please find below the tree:

    
    
    .
    ├── scrapy.cfg
    └── tutorial
        ├── __init__.py
        ├── __init__.pyc
        ├── items.py
        ├── pipelines.py
        ├── settings.py
        ├── settings.pyc
        └── spiders
            ├── __init__.py
            ├── __init__.pyc
            └── dmoz_spider.py
    
    2 directories, 10 files
    

Now when I try to run "scapy crawl dmoz" I get this:

    
    
    $ scrapy crawl dmoz
    
    2013-08-14 12:51:40+0200 [scrapy] INFO: Scrapy 0.16.5 started (bot: tutorial)
    2013-08-14 12:51:40+0200 [scrapy] DEBUG: Enabled extensions: LogStats, TelnetConsole, CloseSpider, WebService, CoreStats, SpiderState
    Traceback (most recent call last):
      File "/Library/Frameworks/Python.framework/Versions/2.7/bin/scrapy", line 5, in <module>
        pkg_resources.run_script('Scrapy==0.16.5', 'scrapy')
      File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pkg_resources.py", line 499, in run_script
        self.require(requires)[0].run_script(script_name, ns)
      File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pkg_resources.py", line 1235, in run_script
        execfile(script_filename, namespace, namespace)
      File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/Scrapy-0.16.5-py2.7.egg/EGG-INFO/scripts/scrapy", line 4, in <module>
        execute()
      File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/Scrapy-0.16.5-py2.7.egg/scrapy/cmdline.py", line 131, in execute
        _run_print_help(parser, _run_command, cmd, args, opts)
      File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/Scrapy-0.16.5-py2.7.egg/scrapy/cmdline.py", line 76, in _run_print_help
        func(*a, **kw)
      File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/Scrapy-0.16.5-py2.7.egg/scrapy/cmdline.py", line 138, in _run_command
        cmd.run(args, opts)
      File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/Scrapy-0.16.5-py2.7.egg/scrapy/commands/crawl.py", line 43, in run
        spider = self.crawler.spiders.create(spname, **opts.spargs)
      File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/Scrapy-0.16.5-py2.7.egg/scrapy/command.py", line 33, in crawler
        self._crawler.configure()
      File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/Scrapy-0.16.5-py2.7.egg/scrapy/crawler.py", line 40, in configure
        self.spiders = spman_cls.from_crawler(self)
      File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/Scrapy-0.16.5-py2.7.egg/scrapy/spidermanager.py", line 35, in from_crawler
        sm = cls.from_settings(crawler.settings)
      File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/Scrapy-0.16.5-py2.7.egg/scrapy/spidermanager.py", line 31, in from_settings
        return cls(settings.getlist('SPIDER_MODULES'))
      File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/Scrapy-0.16.5-py2.7.egg/scrapy/spidermanager.py", line 22, in __init__
        for module in walk_modules(name):
      File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/Scrapy-0.16.5-py2.7.egg/scrapy/utils/misc.py", line 65, in walk_modules
        submod = __import__(fullpath, {}, {}, [''])
      File "/Users//Documents/tutorial/tutorial/spiders/dmoz_spider.py", line 1
        ActivePython 2.7.2.5 (ActiveState Software Inc.) based on
                       ^
    SyntaxError: invalid syntax
    

Does anybody know what is wrong with the steps I'm making? Thank you for your
help. This is my very first programming experience, so it might be a very
stupid issue.


Answer: It's not an indentation problem, the error message is clear:

    
    
    File "/Users//Documents/tutorial/tutorial/spiders/dmoz_spider.py", line 1
    ActivePython 2.7.2.5 (ActiveState Software Inc.) based on
                   ^
    SyntaxError: invalid syntax
    

You clearly have copy pasted the code in IDLE including starting strings from
the IDLE, which are no code.

Instead of copy-pasting, try opening an editor and actually typing the
tutorial code there, you'll learn better and you wont be pasting crap
inadvertently.

