title: How can I convert serialized data in boost::beast to a string so that I could process it in a FIFO manner?
body: <p>I have an application of a client where I need to receive http "long running requests" from a server. I send a command, and after getting the header of the response, I have to just receive json data separated by <code>\r\n</code> until the connection is terminated.</p>
  
  <p>I managed to adapt <a href="https://www.boost.org/doc/libs/1_66_0/libs/beast/example/http/client/async/http_client_async.cpp" rel="nofollow noreferrer">boost beast client example</a> to send the message and receive the header and parse it and receive responses from the server. However, I failed at finding a way to serialize the data so that I could process the json messages.</p>
  
  <p>The closest demonstration of the problem can be found in <a href="https://www.boost.org/doc/libs/1_69_0/libs/beast/doc/html/beast/more_examples/http_relay.html" rel="nofollow noreferrer">this relay example</a>. In that example (<code>p</code> is a parser, <code>sr</code> is a serializer, <code>input</code> is a socket input stream and <code>output</code> is an socket output stream), after reading the http header, we have a loop that reads continuously from the server:</p>
  
  <pre><code>do
  {
      if(! p.is_done())
      {
          // Set up the body for writing into our small buffer
          p.get().body().data = buf;
          p.get().body().size = sizeof(buf);
  
          // Read as much as we can
          read(input, buffer, p, ec);
  
          // This error is returned when buffer_body uses up the buffer
          if(ec == error::need_buffer)
              ec = {};
          if(ec)
              return;
  
          // Set up the body for reading.
          // This is how much was parsed:
          p.get().body().size = sizeof(buf) - p.get().body().size;
          p.get().body().data = buf;
          p.get().body().more = ! p.is_done();
      }
      else
      {
          p.get().body().data = nullptr;
          p.get().body().size = 0;
      }
  
      // Write everything in the buffer (which might be empty)
      write(output, sr, ec);
  
      // This error is returned when buffer_body uses up the buffer
      if(ec == error::need_buffer)
          ec = {};
      if(ec)
          return;
  }
  while(! p.is_done() &amp;&amp; ! sr.is_done());
  </code></pre>
  
  <p>A few things I don't understand here:</p>
  
  <ol>
  <li>We're done reading the header. Why do we need boost beast and not boost asio to read a raw tcp message? When I tried to do that (with both async_read/async_read_some) I got an infinite reads of zero size.</li>
  <li>The documentation of <a href="https://www.boost.org/doc/libs/develop/libs/beast/doc/html/beast/ref/boost__beast__http__parser.html" rel="nofollow noreferrer">parser</a> says (at the end of the page) that a new instance is needed for every message, but I don't see that in the example.</li>
  <li>Since tcp message reading is not working, is there a way to convert the parser/serializer data to some kind of string? Even write it to a text file in a FIFO manner, so that I could process it with some json library? I don't want to use another socket like the example.</li>
  </ol>
  
  <p>The function <code>boost::beast::buffers()</code> failed to compile for the parser and the serializer, and for the parser there's no consume function, and the serializer's consume seems to be for particular http parts of the message, which fires an assert if I do it for <code>body()</code>.</p>
  
  <p>Besides that, I also failed at getting consistent chunks of data from the parser and the buffer with old-school <code>std::copy</code>. I don't seem to understand how to combine the data together to get the stream of data. Consuming the buffer with <code>.consume()</code> at any point while receiving data leads to <code>need buffer</code> error.</p>
  
  <p>I would really appreciate someone explaining the logic of how all this should work together.</p>