title: Scrapy error IgnoreRequest
body: <p>I have error <code>scrapy.exceptions.IgnoreRequest</code> when I want to get data from this link <a href="http://edirectory.matrade.gov.my/application/edirectory.nsf/category?OpenForm&amp;query=product&amp;code=PT&amp;sid=BED1E22D5BE3F9B5394D6AF0E742828F" rel="nofollow noreferrer">link</a>. I 'm a newbie in Scrapy, I can get data from the other links but with this link, I cannot get data. I don't know why, I search and find out the error in Exceptions but how to fix this<a href="https://doc.scrapy.org/en/latest/topics/exceptions.html" rel="nofollow noreferrer">exception</a>:</p>
  
  <pre><code>2017-02-16 23:05:39 [scrapy.core.engine] DEBUG: Crawled (200) &lt;GET http://edirectory.matrade.gov.my/robots.txt&gt; (referer: None)
  2017-02-16 23:05:39 [scrapy.downloadermiddlewares.robotstxt] DEBUG: Forbidden by robots.txt: &lt;GET http://edirectory.matrade.gov.my/application/edirectory.nsf/category?
  
      OpenForm&amp;query=product&amp;code=PT&amp;sid=BED1E22D5BE3F9B5394D6AF0E742828F&gt;
      2017-02-16 23:05:39 [scrapy.core.engine] INFO: Closing spider (finished)
      2017-02-16 23:05:39 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
      {'downloader/exception_count': 1,
       'downloader/exception_type_count/scrapy.exceptions.IgnoreRequest': 1,
       'downloader/request_bytes': 233,
       'downloader/request_count': 1,
       'downloader/request_method_count/GET': 1,
       'downloader/response_bytes': 215,
       'downloader/response_count': 1,
       'downloader/response_status_count/200': 1,
       'finish_reason': 'finished',
       'finish_time': datetime.datetime(2017, 2, 16, 16, 5, 39, 373000),
       'log_count/DEBUG': 3,
       'log_count/INFO': 7,
       'response_received_count': 1,
       'scheduler/dequeued': 1,
       'scheduler/dequeued/memory': 1,
       'scheduler/enqueued': 1,
       'scheduler/enqueued/memory': 1,
       'start_time': datetime.datetime(2017, 2, 16, 16, 5, 33, 557000)}
      2017-02-16 23:05:39 [scrapy.core.engine] INFO: Spider closed (finished)
  </code></pre>
  
  <p>Thanks in advance!
  (My girl friend needs my help, she thinks I can do everything with all links :()</p>