5 examples of reading, writing and pipelining streams in NodeJS

1. Piping input to output stream (file examples)

const fs = require('fs'); let input = fs.createReadStream('/var/www/examples/test.txt'); let output = fs.createWriteStream('/tmp/out.txt'); input.pipe(output); 

• require('fs') - library to work with file system,
• fs.createReadStream( - create stream to read data from (file in our case),
• fs.createWriteStream - open file and create writing stream from it,
• /var/www/examples/test.txt - path to file to stream read,
• /tmp/out.txt - path to file to write stream to,
• .pipe( - pipe input stream to output (in our case - read from input file and write to output).

Open original or edit on Github.

2. Preferred way to pipeline streams

const fs = require('fs'); const { pipeline } = require('stream'); let input = fs.createReadStream('/var/www/examples/test.txt'); let output = fs.createWriteStream('/tmp/out.txt'); pipeline(input, output, (err) => console.log(err)); 

• require('fs') - library to work with file system,
• fs.createReadStream( - create stream to read data from (file in our case),
• fs.createWriteStream - open file and create writing stream from it,
• /var/www/examples/test.txt - path to file to stream read,
• /tmp/out.txt - path to file to write stream to,
• pipeline - pipelines given streams (left to right) and properly destroys all objects after processing,
• (err) => console.log(err) - last argument of pipeline() is always error callback.

Open original or edit on Github.

3. How to read input stream to Buffer

const fs = require('fs'); let input = fs.createReadStream('/var/www/examples/test.txt'); input.on('data', buf => { console.log(buf); }); 

• require('fs') - library to work with file system,
• fs.createReadStream( - create stream to read data from (file in our case),
• /var/www/examples/test.txt - path to file to stream read,
• .on('data' - handle date reading from stream,
• buf - buffer gets data chunk read from stream.

Open original or edit on Github.

4. How to read input stream to string

const fs = require('fs'); let input = fs.createReadStream('/var/www/examples/test.txt'); const chunks = []; input.on('data', buf => chunks.push(buf)); input.on('end', () => console.log( Buffer.concat(chunks).toString() )) 

• require('fs') - library to work with file system,
• fs.createReadStream( - create stream to read data from (file in our case),
• /var/www/examples/test.txt - path to file to stream read,
• .on('data' - handle date reading from stream,
• chunks.push(buf) - push each chunk into array,
• Buffer.concat(chunks) - join all chunks into single buffer,
• .toString() - convert Buffer to string,
• input.on('end' - fire when stream reading was finished.

Open original or edit on Github.

5. How to transform streams using Transform interface

const fs = require('fs'); const { Transform } = require("stream"); let input = fs.createReadStream('/var/www/examples/test.txt'); const my_transform = new Transform({ transform(chunk, encoding, callback) { callback(null, 'TRANSFORMED: ' + chunk.toString()); }, }); my_transform.on('data', buf => console.log(buf.toString())); input.pipe(my_transform); 

• require('fs') - library to work with file system,
• fs.createReadStream( - create stream to read data from (file in our case),
• new Transform( - create new stream that transforms data,
• transform(chunk, encoding, callback) - chunk transformation function,
• 'TRANSFORMED: ' + chunk.toString() - return transformed chunk to callback (we add TRANSFORMED: text to chunk as transformation example),
• .on('data' - handle date reading from stream,
• .pipe( - pipe object to the given stream (read from file and pipe to transforming stream).

Open original or edit on Github.