title: Why am I getting "(304) Not Modified" error on some links when using HttpWebRequest?
body: <p>Any ideas why on some links that I try to access using HttpWebRequest I am getting "The remote server returned an error: (304) Not Modified." in the code?</p>
  
  <p>The code I'm using is from <a href="http://www.jeffcrossman.com/2009/08/27/using-httplistener-to-build-a-proxy/" rel="noreferrer">Jeff's post here</a> (the page seems to have disappeared, see an <a href="http://web.archive.org/web/20091010045702/http://www.jeffcrossman.com/2009/08/27/using-httplistener-to-build-a-proxy" rel="noreferrer">archive copy at the Wayback Machine</a>).</p>
  
  <p>Note the concept of the code is a simple proxy server, so I'm pointing my browser at this locally running piece of code, which gets my browsers request, and then proxies it on by creating a new HttpWebRequest, as you'll see in the code.  It works great for most sites/links, but for some this error comes up.  You will see one key bit in the code is where it seems to copy the http header settings from the browser request to it's request out to the site, and it copies in the header attributes.  Not sure if the issue is something to do with how it mimics this aspect of the request and then what happens as the result comes back?</p>
  
  <pre><code>case "If-Modified-Since":
     request.IfModifiedSince = DateTime.Parse(listenerContext.Request.Headers[key]);
     break;
  </code></pre>
  
  <p>I get the issue for example from <a href="http://en.wikipedia.org/wiki/Main_Page" rel="noreferrer">http://en.wikipedia.org/wiki/Main_Page</a></p>
  
  <p><strong><em>PS. UPDATE HERE</em></strong> </p>
  
  <p>Still can't work this out.  Basically I can identify 1 link which has an issue, and it seems to work fine, 2nd time it gets the error, 3rd time OK, 4th time gets the error, 5th time OK etc.  As if there is some state not getting cleared or something in the code.  I've tried to clean up the code a bit using "using" type statements etc.</p>
  
  <p>Here's the code.  If anyone can spot why every 2nd time I browse to a link like <a href="http://newsimg.bbc.co.uk/css/screen/1_0_16/nol/v4/story.css" rel="noreferrer">http://newsimg.bbc.co.uk/css/screen/1_0_16/nol/v4/story.css</a> (starting at the 2nd time, not the first) via this proxy code I get the error I'd love to hear.</p>
  
  <pre><code>class Program
  {
      static void Main(string[] args)
      {
          Proxy p = new Proxy(8080);
  
          Thread proxythread = new Thread(new ThreadStart(p.Start));
          proxythread.Start();
  
          Console.WriteLine("Proxy Started. Press Any Key To Stop...");
          Console.ReadKey();
  
          p.Stop();
       }
  }
  
  public class Proxy
  {
      private HttpListener _listener;
      private int _port;
  
      public Proxy(int port)
      {
          int defaultport = 8080;
  
          // Setup Thread Pool
          System.Threading.ThreadPool.SetMaxThreads(50, 1000);
          System.Threading.ThreadPool.SetMinThreads(50, 50);
  
          // Sanitize Port Number
          if (port &lt; 1024 || port &gt; 65535)
              port = defaultport;
  
          // Create HttpListener Prefix
          string prefix = string.Format("http://*:{0}/", port);
          _port = port;
  
          // Create HttpListener
          _listener = new HttpListener();
          _listener.Prefixes.Add(prefix);
      }
  
      public void Start()
      {
          _listener.Start();
  
          while (true)
          {
              HttpListenerContext request = null;
  
              try
              {
                  request = _listener.GetContext();
  
                  // Statistics (by Greg)
                  int availThreads = -1;
                  int compPortThreads = -1;
                  ThreadPool.GetAvailableThreads(out availThreads, out compPortThreads);
                  log("INFO", request.Request.Url.ToString(), "START - [" + availThreads + "]");
  
                  ThreadPool.QueueUserWorkItem(ProcessRequest, request);
              }
              catch (HttpListenerException ex)
              {
                  log("ERROR", "NA", "INFO: HttpListenerException - " + ex.Message);
                  break;
              }
              catch (InvalidOperationException ex)
              {
                  log("ERROR", "NA", "INFO: InvalidOperationException - " + ex.Message);
                  break;
              }
          }
      }
  
      public void Stop()
      {
          _listener.Stop();
      }
  
      private void log(string sev, string uri, string message)
      {
          Console.Out.WriteLine(Process.GetCurrentProcess().Id + " - " + sev + " (" + uri + "): " + message);
      }
  
      private void ProcessRequest(object _listenerContext)
      {
          #region local variables
          HttpWebRequest psRequest;                   // Request to send to remote web server
          HttpWebResponse psResponse;                 // Response from remote web server         
          List&lt;byte&gt; requestBody = new List&lt;byte&gt;();  // Byte array to hold the request's body
          List&lt;byte&gt; responseBody = new List&lt;byte&gt;(); // Byte array to hold the response's body
          byte[] buffer;
          string uri = "";
          #endregion
  
          var listenerContext = (HttpListenerContext)_listenerContext;
          uri = listenerContext.Request.Url.ToString().Replace(string.Format(":{0}", _port), "");
  
          // Create Interent Request 
          HttpWebRequest internetRequest = (HttpWebRequest)WebRequest.Create(uri);
          #region Build Request Up
          internetRequest.Method = listenerContext.Request.HttpMethod;
          internetRequest.ProtocolVersion = listenerContext.Request.ProtocolVersion;
          internetRequest.UserAgent = listenerContext.Request.UserAgent;
          foreach (string key in listenerContext.Request.Headers.AllKeys)
          {
              try
              {
                  switch (key)
                  {
                      case "Proxy-Connection":
                      case "Connection":
                          internetRequest.KeepAlive = (listenerContext.Request.Headers[key].ToLower() == "keep-alive") ? true : false;
                          break;
  
                      case "Content-Length":
                          internetRequest.ContentLength = listenerContext.Request.ContentLength64;
                          break;
  
                      case "Content-Type":
                          internetRequest.ContentType = listenerContext.Request.ContentType;
                          break;
  
                      case "Accept":
                          internetRequest.Accept = listenerContext.Request.Headers[key];
                          break;
  
                      case "Host":
                          break;
  
                      case "Referer":
                          internetRequest.Referer = listenerContext.Request.Headers[key];
                          break;
  
                      case "If-Modified-Since":
                          internetRequest.IfModifiedSince = DateTime.Parse(listenerContext.Request.Headers[key]);
                          break;
  
                      default:
                          internetRequest.Headers.Add(key, listenerContext.Request.Headers[key]);
                          break;
                  }
              }
              catch (Exception ex)
              {
                  Console.WriteLine("Error settup up psRequest object. Error = " + ex.Message + "\n" + ex.StackTrace);
              }
          }
          #endregion
  
          #region Copy content into request
          buffer = new byte[1024];
          using (Stream instream = listenerContext.Request.InputStream)
          {
              int incount = instream.Read(buffer, 0, buffer.Length);
              while (incount &gt; 0)
              {
                  internetRequest.GetRequestStream().Write(buffer, 0, incount);
                  incount = instream.Read(buffer, 0, buffer.Length);
              }
          }
          #endregion
  
          // Get Internet Response
          HttpWebResponse internetResponse = null;
          try
          {
              using (internetResponse = (HttpWebResponse)internetRequest.GetResponse())
              {
                  #region Configure Local Response Header Keys
                  foreach (string key in internetResponse.Headers.Keys)
                  {
                      try
                      {
                          switch (key)
                          {
                              case "Transfer-Encoding":
                                  listenerContext.Response.SendChunked = (internetResponse.Headers[key].ToLower() == "chunked") ? true : false;
                                  break;
  
                              case "Content-Length":
                                  listenerContext.Response.ContentLength64 = internetResponse.ContentLength;
                                  break;
  
                              case "Content-Type":
                                  listenerContext.Response.ContentType = internetResponse.Headers[key];
                                  break;
  
                              case "Keep-Alive":
                                  listenerContext.Response.KeepAlive = true;
                                  break;
  
                              default:
                                  listenerContext.Response.Headers.Add(key, internetResponse.Headers[key]);
                                  break;
                          }
                      }
                      catch (Exception ex)
                      {
                          log("ERROR", uri, "Error settup up listenerContext.Response objects. Error = " + ex.Message + "\n" + ex.StackTrace);
                      }
                  }
                  #endregion
  
                  try
                  {
                      // Transfer the body data from Internet Response to Internal Response
                      buffer = new byte[1024];
                      using (Stream inputStream = internetResponse.GetResponseStream())
                      {
                          int outcount = inputStream.Read(buffer, 0, buffer.Length);
                          while (outcount &gt; 0)
                          {
                              listenerContext.Response.OutputStream.Write(buffer, 0, outcount);
                              outcount = inputStream.Read(buffer, 0, buffer.Length);
                          }
                      }
                  }
                  catch (Exception ex)
                  {
                      log("ERROR", uri, "Could not obtain response from URI: " + ex.Message);
                  }
                  finally
                  {
                      listenerContext.Response.OutputStream.Close();
                  }
              }
          }
          catch (Exception ex)
          {
              //if (ex is InvalidOperationException ||
              //    ex is ProtocolViolationException ||
              //    ex is WebException)
              //{
              //    log(uri, "Could not successfully get response: " + ex.GetType() + " - " + ex.Message);
              //    listenerContext.Response.Close();
              //    return;
              //}
              //else { throw; }
  
              log("ERROR", uri, "Could not successfully get response: " + ex.GetType() + " - " + ex.Message);
              listenerContext.Response.Close();
          }
      }
  }
  </code></pre>
  
  <p>And here is an example of what I see - first hit is good, 2nd has error...</p>
  
  <pre><code>Proxy Started. Press Any Key To Stop...
  2080 - INFO (http://newsimg.bbc.co.uk:8080/css/screen/1_0_16/nol/v4/story.css): START - [50]
  2080 - INFO (http://newsimg.bbc.co.uk:8080/css/screen/1_0_16/nol/v4/story.css): START - [50]
  2080 - ERROR (http://newsimg.bbc.co.uk/css/screen/1_0_16/nol/v4/story.css): Could not successfully get response: System.Net.WebException - The remote server returned an error: (304) Not Modified.
  </code></pre>