title: Perl LWP get blocked by server 404 not found
body: <p>I am using <code>LWP::UserAgent</code> to download an csv in unix server</p>
  
  <pre><code>my $ua1 = LWP::UserAgent-&gt;new();
  my $res = $ua1-&gt;get($equity_history_url, @netscape_like_headers);
  </code></pre>
  
  <p>The server keeps giving me an error with code 404 (not found)</p>
  
  <p>Despite that I can download this file from the browser -> 
  <a href="http://www.nseindia.com/content/equities/scripvol/datafiles/18-08-2013-TO-17-08-2015ADANIPOWERALLN.csv" rel="nofollow">http://www.nseindia.com/content/equities/scripvol/datafiles/18-08-2013-TO-17-08-2015ADANIPOWERALLN.csv</a></p>
  
  <p>and the code works with other pages</p>
  
  <p>I think the issue is one of the <a href="http://www.perlmonks.org/bare/?node_id=251057" rel="nofollow">below</a></p>
  
  <ul>
  <li>The site is looking for a cookie. </li>
  <li>The site is decoding the agent string. </li>
  <li>The site is making a decision based on some other part of the
  HTTP request header.</li>
  </ul>
  
  <p>I tried passing a header similar to my browser which I captured using wireshark</p>
  
  <pre><code>my @netscape_like_headers = (
      'User-Agent' =&gt; 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/44.0.2403.155 Safari/537.36',
      'Accept-Language' =&gt; 'en-US,en;q=0.8',
      'Accept-Charset' =&gt; 'iso-8859-1,*,utf-8',
      'Accept-Encoding' =&gt; 'gzip, deflate, sdch',
      'Upgrade-Insecure-Requests' =&gt; '1',
      'Accept' =&gt;     'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8',
      'Connection' =&gt; 'keep-alive'
  );
  </code></pre>
  
  <p>still no luck. any suggestions?</p>