title: InvalidDataException: Multipart body length limit 16384 exceeded
body: <p><strong>What I am trying to do:</strong></p>
  
  <p>I am attempting to upload a <code>multipart/form-data</code> with a file and a JSON blob using Postman to an ASP.NET Core 2.2 <code>APIController</code> and stream the file to a temporary file on the disk instead of fully into memory due to the files likely being potentially large in size (20MB - 2GB). I've followed both examples from <a href="https://docs.microsoft.com/en-us/aspnet/core/mvc/models/file-uploads?view=aspnetcore-2.2" rel="noreferrer">https://docs.microsoft.com/en-us/aspnet/core/mvc/models/file-uploads?view=aspnetcore-2.2</a> starting with the large file example, but I've also tried testing the small file example with the same error, similar but different stack traces. The server uses Kestrel.</p>
  
  <p><strong>Large file example's stack-trace (Caught in debugger):</strong></p>
  
  <pre><code>Exception has occurred: CLR/System.IO.InvalidDataException
  Exception thrown: 'System.IO.InvalidDataException' in System.Private.CoreLib.dll: 'Multipart body length limit 16384 exceeded.'
     at Microsoft.AspNetCore.WebUtilities.MultipartReaderStream.UpdatePosition(Int32 read)
     at Microsoft.AspNetCore.WebUtilities.MultipartReaderStream.&lt;ReadAsync&gt;d__36.MoveNext()
     at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
     at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
     at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
     at Microsoft.AspNetCore.WebUtilities.StreamHelperExtensions.&lt;DrainAsync&gt;d__3.MoveNext()
     at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
     at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
     at Microsoft.AspNetCore.WebUtilities.MultipartReader.&lt;ReadNextSectionAsync&gt;d__20.MoveNext()
     at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
     at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
     at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
     at LookupServiceAPI.Helpers.FileStreamingHelper.&lt;StreamFile&gt;d__1.MoveNext() in &lt;hidden-path-to-project&gt;\Helpers\FileStreamingHelper.cs:line 35
  </code></pre>
  
  <p><strong>Small file example's stack-trace (returned in response, doesn't hit any breakpoints or debugger exception catches):</strong></p>
  
  <pre><code>System.IO.InvalidDataException: Multipart body length limit 16384 exceeded.
     at Microsoft.AspNetCore.WebUtilities.MultipartReaderStream.UpdatePosition(Int32 read)
     at Microsoft.AspNetCore.WebUtilities.MultipartReaderStream.ReadAsync(Byte[] buffer, Int32 offset, Int32 count, CancellationToken cancellationToken)
     at Microsoft.AspNetCore.WebUtilities.StreamHelperExtensions.DrainAsync(Stream stream, ArrayPool`1 bytePool, Nullable`1 limit, CancellationToken cancellationToken)
     at Microsoft.AspNetCore.WebUtilities.MultipartReader.ReadNextSectionAsync(CancellationToken cancellationToken)
     at Microsoft.AspNetCore.Http.Features.FormFeature.InnerReadFormAsync(CancellationToken cancellationToken)
     at Microsoft.AspNetCore.Mvc.ModelBinding.FormValueProviderFactory.AddValueProviderAsync(ValueProviderFactoryContext context)
     at Microsoft.AspNetCore.Mvc.ModelBinding.CompositeValueProvider.CreateAsync(ActionContext actionContext, IList`1 factories)
     at Microsoft.AspNetCore.Mvc.ModelBinding.CompositeValueProvider.CreateAsync(ControllerContext controllerContext)
     at Microsoft.AspNetCore.Mvc.Internal.ControllerBinderDelegateProvider.&lt;&gt;c__DisplayClass0_0.&lt;&lt;CreateBinderDelegate&gt;g__Bind|0&gt;d.MoveNext()
  --- End of stack trace from previous location where exception was thrown ---
     at Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker.InvokeInnerFilterAsync()
     at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.InvokeNextResourceFilter()
     at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.Rethrow(ResourceExecutedContext context)
     at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.Next(State&amp; next, Scope&amp; scope, Object&amp; state, Boolean&amp; isCompleted)
     at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.InvokeFilterPipelineAsync()
     at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.InvokeAsync()
     at Microsoft.AspNetCore.Routing.EndpointMiddleware.Invoke(HttpContext httpContext)
     at Microsoft.AspNetCore.Routing.EndpointRoutingMiddleware.Invoke(HttpContext httpContext)
     at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.Invoke(HttpContext context)
  </code></pre>
  
  <p><strong>Here is my basic controller code and helper classes for the large file example:</strong></p>
  
  <p>FileStreamingHelper.cs</p>
  
  <pre class="lang-cs prettyprint-override"><code>using System;
  using System.Globalization;
  using System.IO;
  using System.Text;
  using System.Threading.Tasks;
  using Microsoft.AspNetCore.Http;
  using Microsoft.AspNetCore.Http.Features;
  using Microsoft.AspNetCore.Http.Extensions;
  using Microsoft.AspNetCore.Mvc.ModelBinding;
  using Microsoft.AspNetCore.WebUtilities;
  using Microsoft.Net.Http.Headers;
  
  namespace LookupServiceAPI.Helpers
  {
      public static class FileStreamingHelper
      {
          private static readonly FormOptions _defaultFormOptions = new FormOptions();
  
          public static async Task&lt;FormValueProvider&gt; StreamFile(this HttpRequest request, Stream targetStream)
          {
              if (!MultipartRequestHelper.IsMultipartContentType(request.ContentType))
              {
                  throw new Exception($"Expected a multipart request, but got {request.ContentType}");
              }
  
              // Used to accumulate all the form url encoded key value pairs in the 
              // request.
              var formAccumulator = new KeyValueAccumulator();
  
              var boundary = request.GetMultipartBoundary();
              var reader = new MultipartReader(boundary, request.Body);
              reader.BodyLengthLimit = Int32.MaxValue;
              reader.HeadersLengthLimit = Int32.MaxValue;
              var section = await reader.ReadNextSectionAsync(); //EXCEPTION HERE
              while (section != null)
              {
                  ContentDispositionHeaderValue contentDisposition;
                  var hasContentDispositionHeader = ContentDispositionHeaderValue.TryParse(section.ContentDisposition, out contentDisposition);
  
                  if (hasContentDispositionHeader)
                  {
                      if (MultipartRequestHelper.HasFileContentDisposition(contentDisposition))
                      {
                          await section.Body.CopyToAsync(targetStream);
                      }
                      else if (MultipartRequestHelper.HasFormDataContentDisposition(contentDisposition))
                      {
                          // Content-Disposition: form-data; name="key"
                          //
                          // value
  
                          // Do not limit the key name length here because the 
                          // multipart headers length limit is already in effect.
                          var key = HeaderUtilities.RemoveQuotes(contentDisposition.Name);
                          var encoding = GetEncoding(section);
                          using (var streamReader = new StreamReader(
                              section.Body,
                              encoding,
                              detectEncodingFromByteOrderMarks: true,
                              bufferSize: 1024,
                              leaveOpen: true))
                          {
                              // The value length limit is enforced by MultipartBodyLengthLimit
                              var value = await streamReader.ReadToEndAsync();
                              if (String.Equals(value, "undefined", StringComparison.OrdinalIgnoreCase))
                              {
                                  value = String.Empty;
                              }
                              formAccumulator.Append(key.Value, value); // For .NET Core &lt;2.0 remove ".Value" from key
  
                              if (formAccumulator.ValueCount &gt; _defaultFormOptions.ValueCountLimit)
                              {
                                  throw new InvalidDataException($"Form key count limit {_defaultFormOptions.ValueCountLimit} exceeded.");
                              }
                          }
                      }
                  }
  
                  // Drains any remaining section body that has not been consumed and
                  // reads the headers for the next section.
                  section = await reader.ReadNextSectionAsync();
              }
  
              // Bind form data to a model
              var formValueProvider = new FormValueProvider(
                  BindingSource.Form,
                  new FormCollection(formAccumulator.GetResults()),
                  CultureInfo.CurrentCulture);
  
              return formValueProvider;
          }
  
          private static Encoding GetEncoding(MultipartSection section)
          {
              MediaTypeHeaderValue mediaType;
              var hasMediaTypeHeader = MediaTypeHeaderValue.TryParse(section.ContentType, out mediaType);
              // UTF-7 is insecure and should not be honored. UTF-8 will succeed in 
              // most cases.
              if (!hasMediaTypeHeader || Encoding.UTF7.Equals(mediaType.Encoding) || mediaType.Encoding == null)
              {
                  return Encoding.UTF8;
              }
              return mediaType.Encoding;
          }
      }
  }
  </code></pre>
  
  <p>MultipartRequestHelper.cs</p>
  
  <pre><code>using System;
  using System.IO;
  using Microsoft.Net.Http.Headers;
  
  namespace LookupServiceAPI.Helpers
  {
      public static class MultipartRequestHelper
      {
          public static bool IsMultipartContentType(string contentType)
          {
              return !string.IsNullOrEmpty(contentType)
                     &amp;&amp; contentType.IndexOf("multipart/", StringComparison.OrdinalIgnoreCase) &gt;= 0;
          }
  
          public static bool HasFormDataContentDisposition(ContentDispositionHeaderValue contentDisposition)
          {
              // Content-Disposition: form-data; name="key";
              return contentDisposition != null
                     &amp;&amp; contentDisposition.DispositionType.Equals("form-data")
                     &amp;&amp; string.IsNullOrEmpty(contentDisposition.FileName.Value)
                     &amp;&amp; string.IsNullOrEmpty(contentDisposition.FileNameStar.Value);
          }
  
          public static bool HasFileContentDisposition(ContentDispositionHeaderValue contentDisposition)
          {
              // Content-Disposition: form-data; name="myfile1"; filename="Misc 002.jpg"
              return contentDisposition != null
                     &amp;&amp; contentDisposition.DispositionType.Equals("form-data")
                     &amp;&amp; (!string.IsNullOrEmpty(contentDisposition.FileName.ToString())
                         || !string.IsNullOrEmpty(contentDisposition.FileNameStar.ToString()));
          }
      }
  }
  </code></pre>
  
  <p>Minimal Controller:</p>
  
  <pre><code>[Route("api/v0.1/data/excel")]
  [ApiController]
  public class DataExcelController : ControllerBase
  {
      [HttpPost, DisableRequestSizeLimit]
      public async Task&lt;IActionResult&gt; ImportExcel()
      {
          var processID = Guid.NewGuid();
          FormValueProvider multipartContent;
          string tempFilePath = Path.GetTempPath() + processID;
          using(var tempStream = System.IO.File.OpenWrite(tempFilePath))
          {
              multipartContent = await Request.StreamFile(tempStream);
          }
          /** Other unnecessary code **/
          return Ok();
      }
  }
  </code></pre>
  
  <p>Startup.cs</p>
  
  <pre><code>namespace LookupServiceAPI
  {
      public class Startup
      {
          public Startup(IConfiguration configuration)
          {
              Configuration = configuration;
          }
  
          public IConfiguration Configuration { get; }
  
          // This method gets called by the runtime. Use this method to add services to the container.
          public void ConfigureServices(IServiceCollection services)
          {
              services.AddMvc().SetCompatibilityVersion(CompatibilityVersion.Version_2_2);
  
              services.Configure&lt;FormOptions&gt;(x =&gt;
              {
                  x.MultipartHeadersLengthLimit = Int32.MaxValue;
                  x.MultipartBoundaryLengthLimit = Int32.MaxValue;
                  x.MultipartBodyLengthLimit = Int64.MaxValue;
                  x.ValueLengthLimit = Int32.MaxValue;
                  x.BufferBodyLengthLimit = Int64.MaxValue;
                  x.MemoryBufferThreshold = Int32.MaxValue;
              });
          }
  
          // This method gets called by the runtime. Use this method to configure the HTTP request pipeline.
          public void Configure(IApplicationBuilder app, IHostingEnvironment env)
          {
              if (env.IsDevelopment())
              {
                  app.UseDeveloperExceptionPage();
              }
              else
              {
                  // The default HSTS value is 30 days. You may want to change this for production scenarios, see https://aka.ms/aspnetcore-hsts.
                  app.UseHsts();
              }
  
              app.UseHttpsRedirection();
              app.UseMvc();
          }
      }
  }
  </code></pre>
  
  <p>Postman configuration image (only values set are in the image, no values set in Headers):
  <a href="https://i.stack.imgur.com/XRlSy.png" rel="noreferrer"><img src="https://i.stack.imgur.com/XRlSy.png" alt="Postman Configuration Image"></a></p>
  
  <p>Postman Console output:
  <a href="https://i.stack.imgur.com/X3yBN.png" rel="noreferrer"><img src="https://i.stack.imgur.com/X3yBN.png" alt="Postman Console Output"></a></p>
  
  <p><strong>Things I've tried:</strong></p>
  
  <p>From <a href="https://stackoverflow.com/questions/41968462/multipart-body-length-limit-16384-exceeded">Multipart body length limit 16384 exceeded</a>:</p>
  
  <ul>
  <li>Set <code>MemoryBufferThreshold</code></li>
  <li>Set <code>MultipartBodyLengthLimit</code></li>
  <li>Ensured the Postman configured Headers does not have <code>Content-Type</code> manually set to <code>multipart/form-data</code></li>
  </ul>
  
  <p>From <a href="https://stackoverflow.com/questions/40364226/multipart-body-length-limit-exceeded-exception">Multipart body length limit exceeded exception</a>:</p>
  
  <ul>
  <li>Set <code>ValueLengthLimit</code></li>
  <li>Used <code>[DisableRequestSizeLimit]</code></li>
  </ul>
  
  <p><strong>Where I think the problem is, but I'm unsure of a workaround or what's causing the issue:</strong>
  <a href="https://github.com/aspnet/AspNetCore/blob/master/src/Http/WebUtilities/src/MultipartReader.cs#L48-L50" rel="noreferrer">https://github.com/aspnet/AspNetCore/blob/master/src/Http/WebUtilities/src/MultipartReader.cs#L48-L50</a></p>
  
  <p>It seems the preamble for my request is beyond the size limit of <code>1024 * 16</code> (16384) set for <code>DefaultHeadersLengthLimit</code>, but I have no clue why that could be the case. Or, if the preamble should be greater than that, how to work around it without re-implementing the whole set of classes or waiting for Microsoft to put out a fix which doesn't seem to be coming down the pipeline:
  <a href="https://github.com/aspnet/Mvc/issues/7019" rel="noreferrer">https://github.com/aspnet/Mvc/issues/7019</a>
  <a href="https://github.com/aspnet/HttpAbstractions/issues/736" rel="noreferrer">https://github.com/aspnet/HttpAbstractions/issues/736</a></p>
  
  <p><strong>Apparently someone fixed their issue which was very similar to mine (<a href="https://github.com/aspnet/Mvc/issues/5128#issuecomment-307675219" rel="noreferrer">https://github.com/aspnet/Mvc/issues/5128#issuecomment-307675219</a>) here:</strong> <a href="https://github.com/aspnet/Mvc/issues/5128#issuecomment-307962922" rel="noreferrer">https://github.com/aspnet/Mvc/issues/5128#issuecomment-307962922</a> but I can't seem to figure out how to figure out if it even applies.</p>
  
  <p>Hopefully this is enough information. If not, please let me know what you need and I'd be happy to provide it or test out any suggestions. I've been stuck researching this and trying everything I can find for 6+ hours now.</p>