title: Multipart form upload to S3 using libcurl and CURLFORM_STREAM
body: <p>I am trying to upload in C/C++ code to S3 using libcurl. I am uploading binary data to be generated from a callback.</p>
  
  <p>I've been staring at this for a while, so I may be way off :( but here is what my code looks like right now:</p>
  
  <p>First, I build my form like so:</p>
  
  <pre><code>struct curl_httppost *buildForm(void *streamData, long contentLength) {
     struct curl_httppost *formpost=NULL;
     struct curl_httppost *lastptr=NULL;
     for( int i=0; i&lt;form.size(); ++i ) {
       curl_formadd(&amp;formpost,
             &amp;lastptr,
             CURLFORM_COPYNAME, form[i].first.c_str(),
             CURLFORM_COPYCONTENTS, form[i].second.c_str(),
             CURLFORM_END);
    }
    curl_formadd(&amp;formpost,
             &amp;lastptr,
             CURLFORM_COPYNAME, "file",
             CURLFORM_STREAM, streamData,
             CURLFORM_CONTENTSLENGTH, (long) contentLength,
             CURLFORM_FILENAME, "chunk",
             //CURLFORM_CONTENTTYPE, "application/octet-stream",
             CURLFORM_END);
    return formpost;
  }
  </code></pre>
  
  <p>Then I setup the request like this:</p>
  
  <pre><code>  curl_easy_reset( handle );
    struct curl_slist *headers = NULL;
    headers = curl_slist_append(headers, "Expect:");
    curl_easy_setopt( handle, CURLOPT_HTTPHEADER, headers);
    curl_easy_setopt( handle, CURLOPT_VERBOSE, 1L );
    curl_easy_setopt( handle, CURLOPT_DEBUGFUNCTION, curlDebugger );
    curl_easy_setopt( handle, CURLOPT_URL, postInstructions.url.c_str() );
    curl_easy_setopt( handle, CURLOPT_POST, 1L);
  
    // set the data and header read callbacks:
    curl_easy_setopt( handle, CURLOPT_READFUNCTION, &amp;(UploadCallback::writeDataStatic));
    curl_easy_setopt( handle, CURLOPT_HTTPPOST, formpost);
    //I have tried with and without this line
    curl_easy_setopt( handle, CURLOPT_POSTFIELDSIZE_LARGE, (curl_off_t)contentLength);
    curl_easy_setopt( handle, CURLOPT_WRITEFUNCTION, &amp;(XonamiResponse::readDataStatic));
    curl_easy_setopt( handle, CURLOPT_WRITEDATA, &amp;response);
    curl_easy_setopt( handle, CURLOPT_HEADERFUNCTION, &amp;(XonamiResponse::readHeaderStatic));
    curl_easy_setopt( handle, CURLOPT_WRITEHEADER, &amp;response);
  
    // do it!
    int success = curl_easy_perform( handle );
  </code></pre>
  
  <p>However, curl complains:</p>
  
  <pre><code>  operation aborted by callback
  </code></pre>
  
  <p>even though I am generating the same number of bytes as contentLength. When I setup my callback to just fill the buffers with zero as long as data is requested, S3 responds with:</p>
  
  <pre><code>  The body of your POST request is not well-formed multipart/form-data.
  </code></pre>
  
  <p>and looking at the raw form it looks to me like the last (closing) boundary is missing.</p>
  
  <p>curl --version says:</p>
  
  <p>curl 7.19.7 (universal-apple-darwin10.0) libcurl/7.19.7 OpenSSL/0.9.8r zlib/1.2.3
  Protocols: tftp ftp telnet dict ldap http file https ftps 
  Features: GSS-Negotiate IPv6 Largefile NTLM SSL libz </p>