title: Remote server returns 404 when requesting an existing url
body: <p>I've again got a strange problem:</p>
  
  <p>I'm writing a crawler to index a specific site. For some weeks it worked fine and I only ran into problems when sending too many requests per hour.</p>
  
  <p>But now I can't even access a single page.</p>
  
  <p>But what's even stranger: I have to submit some form values via POST, but the server returns a 404 error - although the URL is definitely correct.</p>
  
  <p>I implemented many techniques to prevent beeing recognized as a bot: changing user-agent, delays, and I'm sending a Referer-header to pretend the form was submitted from their own website.</p>
  
  <p>May this again be a Spam- or DDOS-protection on their server? Or are there other possible sources of error?</p>