Title: Request ends before readstream events are handled
Tags: javascript;node.js;fs
Question: I'm attempting to make a nodejs function which reads back data from a file with the following code:
app.post('/DownloadData', function(req, res)
req.on('data', function(data) {
if (fs.existsSync('demoDataFile.dat')) {
var rstream = fs.createReadStream('demoDataFile.dat');
var bufs = [];
rstream.on('data', function(chunk) {
bufs.push(chunk);
console.log("data");
});
rstream.on('end', function() {
downbuf = Buffer.concat(bufs);
console.log(downbuf.length);
});
}
});
req.on('end', function() {
console.log("end length: " + downbuf.length);
res.end(downbuf);
});
req.on('error', function(err)
console.error(err.stack);
});
The problem is, the buffer comes back as empty as the req.on('end' ... is called before any of the rstream.on events ("data" and the length aren't printed in the console until after "end length: " has been printed). Am I handling the events wrong or is there some other issue? Any guidance would be appreciated.
Comment: What is `req`? And is there any other code doing anything with `req`?

Comment: I removed the snippet for a reason: Snippets are for **runnable**-in-browser code. I'm off to remove it again. Just use code blocks for code blocks.

Comment: It's not the problem, but I'm curious why you're waiting to start the read until you get the `data` event from the request, given you're not using the data for anything... Also note the existence of `fs.readFile`. Finally, checking for existence before opening is an anti-pattern: Just do the open, and react to the "not found" error. (All of which is tangential to the question.)

Comment: Since it looks like you're using Express, is there a reason why you're not just using [`res.sendFile()`](http://expressjs.com/en/4x/api.html#res.sendFile)?

Comment: @VicciGarner so you need to process the file data further before sending it back to the client? Otherwise, I don't see why you can't use `res.sendFile`. As an aside: `downbuf` is undeclared, which might cause concurrency issues

Comment: sorry yes, I'll ammend the code snippet above

Comment: @robertklep mainly as I want the data in a buffer that's sent back as the response to an ajax call

Comment: @T.J.Crowder That's a good point, the existence check is just in there from debuggings trying to check if the file actually existed, it is redundant

Comment: @robertklep yes the data needs to be processed, at this stage it's just being sent back to the client for testing. apologies for leaving that out, in my code downbuf is declared as a new Buffer(0) at the top of the function


Here is another answer: Not sure why you're reading from req, because you're not using the body taylorlauren@example.com. Also, because the data event can trigger multiple times, the code you're using to read the file may also get called multiple times, which probably isn't what you want.
Here's what I think you want:
app.post("/DownloadData", function(req, res) {
let stream = fs.createReadStream("demoDataFile.dat");
// Handle error regarding to creating/opening the file stream.
stream.on('error', function(err) {
console.error(err.stack);
res.sendStatus(500);
// Read the file data into memory.
let bufs = [];
stream.on("data", function(chunk) {
bufs.push(chunk);
console.log("data");
}).on("end", function() {
let downbuf = Buffer.concat(bufs);
console.log(downbuf.length);
...process the buffer...
res.end(downbuf);
You have to be aware that this will read the file into memory entirely. If it's a big file, it may require a lot of memory.
Since you don't specify which operations you have to perform on the file data, I can't recommend an alternative, but there are various modules available that can help you process file data in a streaming fashion (i.e. without having to read the file into memory entirely).

{'source': 'StackExchangeDataset', 'subset': 'stackoverflow.com', 'metadata': {'AnswerCount': 1, 'CommentCount': 9, 'ContentLicense': 'CC BY-SA 3.0', 'CreationDate': '2017:00:08/30/17', 'FavoriteCount': None, 'PostTypeId': 1, 'Tags': '<javascript><node.js><fs>', 'Title': 'Request ends before readstream events are handled', 'ViewCount': 118, 'id': 45962173, 'user_AboutMe': None, 'user_AccountId': 11356590.0, 'user_DisplayName': 'Vicci Garner', 'user_Reputation': 45.0, 'user_Views': 16.0, 'user_id': 8326031.0}}