title: Scrapy SgmlLinkExtractor question
body: <p>I am trying to make the SgmlLinkExtractor to work.</p>
  
  <p>This is the signature:</p>
  
  <pre><code>SgmlLinkExtractor(allow=(), deny=(), allow_domains=(), deny_domains=(), restrict_xpaths(), tags=('a', 'area'), attrs=('href'), canonicalize=True, unique=True, process_value=None)
  </code></pre>
  
  <p>I am just using <code>allow=()</code></p>
  
  <p>So, I enter</p>
  
  <pre><code>rules = (Rule(SgmlLinkExtractor(allow=("/aadler/", )), callback='parse'),)
  </code></pre>
  
  <p>So, the initial url is <code>'http://www.whitecase.com/jacevedo/'</code> and I am entering <code>allow=('/aadler',)</code> and expect that 
  <code>'/aadler/'</code> will get scanned as well. But instead, the spider scans the initial url and then closes:</p>
  
  <pre><code>[wcase] INFO: Domain opened
  [wcase] DEBUG: Crawled &lt;/jacevedo/&gt; (referer: &lt;None&gt;)
  [wcase] INFO: Passed NuItem(school=[u'JD, ', u'Columbia Law School, Harlan Fiske Stone Scholar, Parker School Recognition of Achievement in International and Foreign Law, ', u'2005'])
  [wcase] INFO: Closing domain (finished)
  </code></pre>
  
  <p>What am I doing wrong here?</p>
  
  <p>Is there anyone here who used Scrapy successfully who can help me to finish this spider?</p>
  
  <p>Thank you for the help.</p>
  
  <p>I include the code for the spider below:</p>
  
  <pre><code>from scrapy.contrib.spiders import CrawlSpider, Rule
  from scrapy.contrib.linkextractors.sgml import SgmlLinkExtractor
  from scrapy.selector import HtmlXPathSelector
  from scrapy.item import Item
  from Nu.items import NuItem
  from urls import u
  
  class NuSpider(CrawlSpider):
      domain_name = "wcase"
      start_urls = ['xxxxxx/jacevedo/']
  
      rules = (Rule(SgmlLinkExtractor(allow=("/aadler/", )), callback='parse'),)
  
      def parse(self, response):
          hxs = HtmlXPathSelector(response)
  
          item = NuItem()
          item['school'] = hxs.select('//td[@class="mainColumnTDa"]').re('(?&lt;=(JD,\s))(.*?)(\d+)')
          return item
  
  SPIDER = NuSpider()
  </code></pre>
  
  <p>Note: SO will not let me post more than 1 url so substitute the initial url as necessary. Sorry about that.</p>