title: How to create LinkExtractor rule which based on href in Scrapy
body: <p>I am trying to create simple crawler with <code>Scrapy</code> (scrapy.org). As per example <code>item.php</code> is allowed. How I can write rule which allow url which always start with <code>http://example.com/category/</code> but in <code>GET</code> parameter <code>page</code> should be there with any number of digit with other parameter. Order of these parameter is random.
     Please help How I can write such a rule?</p>
  
  <p>Few valid values are:</p>
  
  <ul>
  <li><a href="http://example.com/category/?page=1&amp;sort=a-z&amp;cache=1" rel="nofollow">http://example.com/category/?page=1&amp;sort=a-z&amp;cache=1</a></li>
  <li><a href="http://example.com/category/?page=1&amp;sort=a-z#" rel="nofollow">http://example.com/category/?page=1&amp;sort=a-z#</a></li>
  <li><a href="http://example.com/category/?sort=a-z&amp;page=1" rel="nofollow">http://example.com/category/?sort=a-z&amp;page=1</a></li>
  </ul>
  
  <p>Following is the code:</p>
  
  <pre><code>import scrapy
  from scrapy.contrib.spiders import CrawlSpider, Rule
  from scrapy.contrib.linkextractors import LinkExtractor
  class MySpider(CrawlSpider):
  name = 'example.com'
  allowed_domains = ['example.com']
  start_urls = ['http://www.example.com/category/']
  
  rules = (
      Rule(LinkExtractor(allow=('item\.php', )), callback='parse_item'),
  )
  
  def parse_item(self, response):
      item = scrapy.Item()
      item['id'] = response.xpath('//td[@id="item_id"]/text()').re(r'ID: (\d+)')
      item['name'] = response.xpath('//td[@id="item_name"]/text()').extract()
      item['description'] = response.xpath('//td[@id="item_description"]/text()').extract()
      return item
  </code></pre>