title: NodeJS handling file upload with streaming and asynchronous validation
body: <p>I am writing an expressJS web app that handles file upload in chunks.  The receivable file contains data that I need to process and validate on the fly.</p>
  
  <p>I am handling it like below:</p>
  
  <pre><code>module.exports = function(req, res, next) {
    logger.info("file converter executing...");
  
    req.on('data',function(chunk){
  
      // Read a line from the chunk
      var line = readALine(req, chunk);
      while ( (line = readALine(req, chunk)) !== null) {
  
        // ignore comment lines and blank lines...
        if (line.indexOf("#") === 0 || line.length === 0) continue;  
  
        result = processLine(req, line);
      }
    });
  
    /** EOF: request stream -- move processing to next layer.  **/
    req.on('end', function() {
      console.log("handing control to next middle ware...");
      return next();
    });
  
  };
  </code></pre>
  
  <p>In the processLine() function, I have to validate each line of incoming data. But the problem is that validation can be asynchronous, because I am using trying to check against some values which are originally in DB, but lazy-loaded to Redis Cache or program's internal cache.  So, the validation is asynchronous.</p>
  
  <p>So, while I am processing a chunk of data, I do not want to receive any further events from the stream, until I have finished the validation of current chunk.  </p>
  
  <p>My questions:</p>
  
  <ol>
  <li><p>Am I right in assuming that the pause(), resume() API will fit here?  So, I can pause the stream, until the promise of async validation finishes, and then resume it?  Is it safe?</p></li>
  <li><p>Assuming that the first query is right, if my validation fails, plan to return res.send(some error).  Will it properly clear all the remaining chunks in the request stream, or should I do anything more?</p></li>
  </ol>
  
  <p>Thanks for any help...</p>