title: System.OutOfMemoryException when uploading a large file (~600MB) using .NET 4.0 & HttpClient
body: <blockquote>
    <p>I am getting this error only on my local workstation and prod server.
    In Dev and Cert it is working fine.</p>
    
    <p>local workstation  - 20 GB memory, Win 7 64 bit, IIS Express, VS
    2013 <br>dev, cert &amp; prod - 8 GB memory , 2008 R2 64 Bit, IIS 7.5</p>
  </blockquote>
  
  <p>I have a web api (.net 4.0) which takes the incoming request body and uploads it to a storage server. configured web api as per <a href="http://www.strathweb.com/2012/09/dealing-with-large-files-in-asp-net-web-api/" rel="nofollow">this website</a>. </p>
  
  <p>I have these in my web.config</p>
  
  <pre><code>&lt;system.webServer&gt;
    &lt;security&gt;
      &lt;requestFiltering&gt;
        &lt;requestLimits maxAllowedContentLength="2147483648" /&gt;
      &lt;/requestFiltering&gt;
    &lt;/security&gt;
  &lt;/system.webServer&gt;
  
  &lt;system.web&gt;
    &lt;httpRuntime maxRequestLength="2097152" /&gt;
  &lt;/system.web&gt;
  </code></pre>
  
  <p>I also have an implementation of <strong>IHostBufferPolicySelector</strong> which returns false for PUT &amp; POST requests. so the request to this web api for PUt &amp; POST are not buffered. </p>
  
  <p>for any files &lt; ~350 MB it is working fine. but web api is throwing out of memory exceptions when file size >= ~ 400 MB and this is happening only on Local workstation and Prod server. </p>
  
  <p>Web Api controller calls below code to stream the request to the destination server </p>
  
  <pre><code>public async Task&lt;HttpResponseMessage&gt; StoreObjectAsync(Uri namespaceUrl, string userName, string password, string objectName, Stream objectContent, string contentType = "application/octet-stream", IDictionary&lt;string, string&gt; systemMetadata = null)
          {                
              Uri namespaceRootUrl = Utilities.GetNamespaceRootUrl(namespaceUrl);
              using (var request = new HttpRequestMessage() { Method = HttpMethod.Put })
              {
                  request.RequestUri = Utilities.CreateRequestUri(namespaceRootUrl, objectName);
  
                  request.Content = new StreamContent(objectContent);
                  request.Content.Headers.ContentType = new MediaTypeHeaderValue(contentType);
  
                  HttpResponseMessage response;
                  response = await this.httpClient.SendAsync(request, HttpCompletionOption.ResponseHeadersRead);
  
                  return response;
              }
          }
  </code></pre>
  
  <p>After doing some research online, i understand from <a href="http://forums.asp.net/t/1793884.aspx?HttpClient%20buffering%20when%20POSTing%20data" rel="nofollow">this link</a> &amp; <a href="http://forums.asp.net/post/4954981.aspx" rel="nofollow">this link</a> that HttpClient on .Net 4.0 buffers the request body and because of that behavior it seemed to me that it is throwing outofmemory exception</p>
  
  <p>so I changed my code to below this time using <strong>HttpWebRequest</strong> using which I have the control to specify that request should be streamed but not buffered. </p>
  
  <pre><code>public async Task&lt;HttpResponseMessage&gt; StoreObjectAsync(Uri namespaceUrl, string userName, string password, string objectName, Stream content, long contentLength, string contentType = "application/octet-stream", IDictionary&lt;string, string&gt; systemMetadata = null)
          {            
  
              Uri namespaceRootUrl = Utilities.GetHCPNamespaceRootUrl(namespaceUrl);
  
              HttpWebRequest httpWebRequest = ((HttpWebRequest)WebRequest.Create(requestUri));           
  
              httpWebRequest.Method = "PUT";
              httpWebRequest.KeepAlive = true;
  
              httpWebRequest.AllowWriteStreamBuffering = false;
              httpWebRequest.ContentType = contentType;
              httpWebRequest.ContentLength = contentLength;
  
  
              using (Stream requestStream = await httpWebRequest.GetRequestStreamAsync())
              {
                  await content.CopyToAsync(requestStream);
              }
  
              var webResponse = await httpWebRequest.GetResponseAsync();
              HttpWebResponse httpWebResponse = (HttpWebResponse)webResponse;
              Stream httpWebResponseContent = httpWebResponse.GetResponseStream();
  
              HttpResponseMessage response = new HttpResponseMessage()
              {
                  StatusCode = httpWebResponse.StatusCode,
                  ReasonPhrase = httpWebResponse.StatusDescription,
                  Content = new StreamContent(httpWebResponseContent)
              };
  
              return response;
          }
  </code></pre>
  
  <p>Now it is working fine on my local machine. I am able to upload files around 1GB without any errors or memory exceptions. Havent pushed this to Prod yet. </p>
  
  <p>But I still dont understand why the same code using HttpClient on .net 4.0 worked on Dev and Cert servers but not on Prod and my local. </p>
  
  <p>please help me in understanding</p>
  
  <blockquote>
    <p>How to find out why it worked on Dev and Cert? <br>What system/server
    configurations will affect the memory allocations to this api?</p>
  </blockquote>