title: If I get a 500 internal server error in Scrapy, how do I skip the URL?
body: <p>I'm scraping data off several thousand pages with the general URL of:</p>
  
  <pre><code>http://example.com/database/?id=(some number)
  </code></pre>
  
  <p>where I am running through the id numbers.</p>
  
  <p>I keep encountering huge chunks of URLs that generate a 500 internal server error, and scrapy goes over these chunks several times for some reason. This eats up a lot of time, so I am wondering if there is a way to just move onto the next URL immediately and not have scrapy send requests several times.</p>