title: MessagePack unpack fails with large Metasploit Framework message in C received via Libcurl
body: <p>First off, I used <a href="https://stackoverflow.com/questions/15393838/how-do-i-unpack-and-extract-data-properly-using-msgpack-c">How do I unpack and extract data properly using msgpack-c?</a> to figure out how to properly unpack data since MessagePack's own C API documentation isn't great. Though, <a href="http://wiki.msgpack.org/display/MSGPACK/QuickStart+for+C+Language" rel="nofollow noreferrer">http://wiki.msgpack.org/display/MSGPACK/QuickStart+for+C+Language</a> is okay too.</p>
  
  <p>I would like to note that the code is working fine for most API calls to metasploit, it is only the big one (<code>module.exploits</code> about 16KB returned) that fails.</p>
  
  <p>Like my last question, I'm at work so can't post the actual code. I will try to show accurately what my code is doing though in the following snippet.</p>
  
  <pre><code>// Function declaration from msgpack headers
  bool msgpack_unpack_next(msgpack_unpacked* result, const char* data, size_t len, size_t* off);
  
  // Called by curl after request
  size_t unpack_result(char *ptr, size_t size, size_t nmemb, void *userdata){
      size_t real_size = size * nmemb;
  
      msgpack_unpacked msg;
      msgpack_unpacked_init(&amp;msg);
      // Fails [returns false] when used on "module.exploits" return, not when
      // "module.payloads" is called nor "auth.login".
      msgpack_unpack_next(&amp;msg, ptr, real_size, 0); 
  
  
  }
  
  msgpack_sbuffer* pack_payload(char **data, size_t size){
      // Code that actually packs payload, pretty standard with nothing special
      // Definitely works, have not had any problems with my payloads
  }
  
  
  int main(void)
  {
      CURL *curl;
      CURLcode res;
  
      curl = curl_easy_init();
  
      char *auth_payload_array[3] = {"auth.login", "&lt;username&gt;", "&lt;password&gt;"};
  
      msgpack_sbuffer *auth_payload = pack_payload(auth_payload_array, 3);
      if(curl) {
          curl_easy_setopt(curl, CURLOPT_URL, "https://&lt;IP Address&gt;/api/");
          curl_easy_setopt(curl, CURLOPT_POSTFIELDS, auth_payload-&gt;data);
          curl_easy_setopt(curl, CURLOPT_POSTFIELDSIZE, auth_payload-&gt;size);
          curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER, false);
  
          // Add custom content type header
          struct curl_slist *headers = NULL;
          headers = curl_slist_append(headers, "Content-Type: binary/message-pack");
          curl_easy_setopt(curl, CURLOPT_HTTPHEADER, headers);
          curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, unpack_result);
  
          res = curl_easy_perform(curl);
          if (res != CURLE_OK){
              fprintf(stderr, "curl_easy_perform() failed: %s\n",
                          curl_easy_strerror(res));
              return 1;
          }
  
          char *exploit_payload_array[2] = {"module.exploits", AUTH_TOKEN};
          msgpack_sbuffer *exploit_payload = pack_payload(exploit_payload_array,2);
  
          curl_easy_setopt(curl, CURLOPT_POSTFIELDS, exploit_payload-&gt;data);
          curl_easy_setopt(curl, CURLOPT_POSTFIELDSIZE, exploit_payload-&gt;size);
  
          res = curl_easy_perform(curl);
          if (res != CURLE_OK){
              fprintf(stderr, "curl_easy_perform() failed: %s\n",
                          curl_easy_strerror(res));
              return 1;
          }
      }
  }
  </code></pre>
  
  <p>I'm aware that the above has memory leaks, I didn't feel like typing all the proper destroys/frees since this is just an example. I left out the part of how <code>AUTH_TOKEN</code> is actually grabbed since it shouldn't be the error, as I don't have problems at that point in the code. I also am aware some of the curl request code could be optimized into functions.</p>
  
  <p>Important note is that the ruby gem works fine with this same API call, but it is a C extension that does not use the standard msgpack library, but does the binary parsing itself.</p>
  
  <p>When I write the raw data from the return to a file instead of the stringbuffer examining it manually shows that it appears to be correct msgpack format.</p>
  
  <p>Any ideas what is going wrong here? I'm going to start using gdb to try and track down the problem, along with looking into turning the ruby c extension into a library I can use since it seems to work. Hopefully though, it is something simple and one of you will catch it!</p>