title: Upload a file over 2.15 GB in R
body: <p>I've got a manual process where I'm uploading 5-6 GB file to a web server via curl:</p>
  
  <pre><code>curl -X POST --data-binary @myfile.csv http://myserver::port/path/to/api
  </code></pre>
  
  <p>This process works fine, but I'd love to automate it using R.  The problem is, I either don't know what I'm doing, or the R libraries for curl don't know how to handle files bigger than ~2GB:</p>
  
  <pre><code>library(RCurl)
  postForm(
       "http://myserver::port/path/to/api",
        file = fileUpload(
          filename = path.expand("myfile.csv"),
          contentType = "text/csv"
        ),.encoding="utf-8")
  </code></pre>
  
  <p>Yeilds <code>Error: Internal Server Error</code></p>
  
  <p>httr doesn't work either:</p>
  
  <pre><code>library(httr)
  POST(
        url = "http://myserver:port/path/to/api",
        body = upload_file(
          path =  path.expand("myfile.csv"),
          type = 'text/csv'),
        verbose()
      )
  </code></pre>
  
  <p>Which yields:</p>
  
  <pre><code>Response [http://myserver:port/path/to/api]
    Date: 2015-06-30 11:11
    Status: 400
    Content-Type: &lt;unknown&gt;
  &lt;EMPTY BODY&gt;
  </code></pre>
  
  <p>httr is a little more informative with the <code>verbose()</code> option, telling me:</p>
  
  <pre><code>-&gt; POST http://myserver:port/path/to/api
  -&gt; User-Agent: libcurl/7.35.0 r-curl/0.9 httr/1.0.0
  -&gt; Host: http://myserver::port
  -&gt; Accept-Encoding: gzip, deflate
  -&gt; Accept: application/json, text/xml, application/xml, */*
  -&gt; Content-Type: text/csv
  -&gt; Content-Length: -2147483648
  -&gt; Expect: 100-continue
  -&gt; 
  &lt;- HTTP/1.1 400 Bad Request
  &lt;- Server: Apache-Coyote/1.1
  &lt;- Transfer-Encoding: chunked
  &lt;- Date: Tue, 30 Jun 2015 11:11:11 GMT
  &lt;- Connection: close
  &lt;- 
  </code></pre>
  
  <p>The <code>Content-Length: -2147483648</code> looks suspiciously like a 32 bit integer overflow, so I think this is a bug in httr.  I suspect RCurl is experiencing a similar failure.</p>
  
  <p>I'd really love a minimal wrapper around <code>curl -X POST --data-binary</code>, but barring that, what are my options for uploading fairly large files from R?</p>