title: Scrapy adds %0A to URLs, causing them to fail
body: <p>I'm almost at my wits end with this one. Basically I have a url that seems to be somehow magical. Specifically it's this: </p>
  
  <blockquote>
    <p><a href="https://www.amazon.de/Instant-Video/b?ie=UTF8&amp;node=3010075031" rel="nofollow">https://www.amazon.de/Instant-Video/b?ie=UTF8&amp;node=3010075031</a></p>
  </blockquote>
  
  <p>When I hit it with <code>requests</code>, everything works fine:</p>
  
  <pre><code>import requests
  test = requests.get("https://www.amazon.de/Instant-Video/b?ie=UTF8&amp;node=3010075031")
  &lt;Response [200]&gt;
  </code></pre>
  
  <p>However, when I use <code>scrapy</code>, the following line pops out:</p>
  
  <pre><code>Crawled (404) &lt;GET https://www.amazon.de/Instant-Video/b?ie=UTF8&amp;node=3010075031%0A&gt;
  </code></pre>
  
  <p>I even tried updating my User-Agent string to no avail. Part of me is concerned that the URL-encoding <code>%0A</code> is responsible, but that seems like it would be rather strange, and I can't find any documentation on how to fix it.</p>
  
  <p>For reference, this is how I'm sending the request, though I'm not sure this will add much information:</p>
  
  <pre><code>for url in review_urls:
      yield scrapy.Request(url, callback=self.get_review_urls)
  </code></pre>
  
  <p>It's important to note that this is the exception rather than the rule. Most URLs work unhindered, but these edge cases are not uncommon.</p>