title: Why is mechanize throwing a HTTP 403 error?
body: <p>For some reason I get a <code>HTTP Error 403: Forbidden</code> when I try opening the page <code>http://questionablecontent.net</code>. I used to get a <code>robots.txt</code> error, but that has been solved. Additionally, I can't even find their robots.txt file. </p>
  
  <p>I can still view the webpage from chrome, so what I'm wondering is: does mechanize look differently than chrome even after setting the appropriate headers? </p>
  
  <p>Here is my code (which does not work):</p>
  
  <pre><code>br = mechanize.Browser()
  cj = cookielib.LWPCookieJar()
  br.set_cookiejar(cj)
  br.set_handle_equiv(True)
  br.set_handle_redirect(True)
  br.set_handle_robots(False)
  br.set_handle_refresh(mechanize._http.HTTPRefreshProcessor(), max_time=1)
  br.addheaders = [('User-agent', 'Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.1) Gecko/2008071615 Fedora/3.0.1-1.fc9 Firefox/3.0.1')]   
  </code></pre>
  
  <p>I also tried setting the addheaders to the same headers as my browser (which I found <a href="http://www.whatsmyuseragent.com/" rel="noreferrer">here</a>):</p>
  
  <pre><code>br.addheaders = [('User-agent','Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.72 Safari/537.36')]
  </code></pre>
  
  <p>... but that didn't work either.</p>
  
  <p>Finally, I tried using Selenium and that worked, seeing as it loads the page in chrome and then communicates with Python. However, I still would like to get it working with mechanize. Also, I'm still unsure as to how chrome and mechanize look different to their server.</p>