title: node.js stream array of json to response
body: <p>I have a REST method that should return a JSON array with certain elements that are read from mongodb (using mongoose).</p>
  
  <p>It should be very simple (in the real case the find method has arguments, but that's no the problem): </p>
  
  <pre><code>OutDataModel.find().stream({transform: JSON.stringify}).pipe(res);
  </code></pre>
  
  <p>The problem with this approach is that I don't get a valid JSON as the result is like this :</p>
  
  <pre><code>{"id":"1","score":11}{"id":"2","score":12}{"id":"3","score":13}
  </code></pre>
  
  <p>and I was expecting this :</p>
  
  <pre><code>[{"id":"1","score":11},{"id":"2","score":12},{"id":"3","score":13}]
  </code></pre>
  
  <p>I have not been able to find a solution, but I am pretty sure there will be a simple one.</p>
  
  <p><strong>What I've tried :</strong></p>
  
  <p>Nothing that I'm proud of but here it goes.</p>
  
  <ol>
  <li>before streaming write <code>'['</code> to the response.</li>
  <li>Instead of JSON.stringify I provided another method that calls JSON.stringify and adds a <code>','</code> at the end</li>
  <li>in the 'end' event of the stream, I write <code>']'</code> to the response.</li>
  </ol>
  
  <p>Still is not working with this "solution" as I have a trailing comma at the end like this : </p>
  
  <pre><code> [{"id":"1","score":11},{"id":"2","score":12},{"id":"3","score":13},]
  </code></pre>
  
  <p>As I said I am pretty sure there should be a clean solution as it is something that should be quite common.</p>
  
  <p>There will be a lot of concurrent invocations to this method, so I don't want to read everything into memory and then write everything to the response. Each of the invocations will not return a lot of records, but all of them together can be huge.  The consumer is a java application with spring, using jackson to parse the JSON.</p>
  
  <p>Please let me know how to do it.</p>
  
  <p><strong>ANSWER</strong></p>
  
  <p>I got it working, by creating a Transform stream as was suggested in the accepted answer.</p>
  
  <p>My stream looks like this :</p>
  
  <pre><code>var arraystream = new stream.Transform({objectMode: true});
  arraystream._hasWritten = false;
  
  
  arraystream._transform = function (chunk, encoding, callback) {
      console.log('_transform:' + chunk);
      if (!this._hasWritten) {
          this._hasWritten = true;
          this.push('[' + JSON.stringify(chunk));
  
      } else {
          this.push(',' + JSON.stringify(chunk));
      }
      callback();
  };
  
  arraystream._flush = function (callback) {
      console.log('_flush:');
      this.push(']');
      callback();
  
  };
  </code></pre>
  
  <p>and the code to use it : </p>
  
  <pre><code>OutDataModel.find().stream().pipe(arraystream).pipe(res);
  </code></pre>
  
  <p>Thanks.</p>