Node. js Streams File Read and Write Operation Detailed Explanation

Node. js is inherently asynchronous and event-driven, making it ideal for handling I/O-related tasks. If you are dealing with I/O-related operations in your application, you can use the stream (stream) in Node. js. So let's take a look at the flows in detail and see how they simplify I/O operations.

What is a stream
Streams are unix pipes that allow you to easily read data from a data source and then flow to another destination.
Simply put, a stream is nothing special, it's just an EventEmitter that implements a few methods. Depending on the method it implements, the stream can become a readable stream (Readable), a writable stream (Writable), or a bidirectional stream (Duplex, both readable and writable).
Readable streams allow you to read data from one data source, while writable streams allow you to write data to a destination.
If you've already used Node. js, you've probably already encountered a stream.
For example, in an HTTP server of Node. js, request is a readable stream and response is a writable stream.
You may also have used fs module, which can help you handle readable and writable streams.
Now let's learn some basics and understand different types of flows. Readable flow and writable flow will be discussed in this article. Bidirectional flow is beyond the scope of this article, so we will not discuss it.
Readable Stream (Readable Streams)
We can use a readable stream to read data from a data source, which can be anything, such as a file in the system, buffer in memory, or even other streams. Because the streams are EventEmitter, they send data with various events. We will use these events to make the flow work.

Read data from a stream
The best way to read data from the stream is to listen for data events and add a callback function. When a data stream comes in, the readable stream sends an data event and the callback function fires. Take a look at the following code snippet:

 var fs = require('fs');
var readableStream = fs.createReadStream('file.txt');
var data = '';

var readableStream.on('data', function(chunk){
 data += chunk;
});

readableStream.on('end', function(){
 console.log(data);
});
 

fs. createReadStream will give you a readable stream.
At the beginning, this flow is not flow dynamic. When you add data's event listener and add a callback function, it will become stream dynamic. After that, it will read 1 small piece of data and pass it to your callback function.
The implementer of the stream determines how often the data event is triggered, for example, HTTP request triggers the data event when several KB data are read. When you read data from a file, you may decide to trigger the data event when a line is read.
When there is no data to read (when the end of the file is read), the stream sends an end event. In the above example, we listened to this event and when we finished reading the file, we printed out the data.
There is another way to read a stream. You just keep calling the read () method in the stream instance before reading the end of the file.

 var fs = require('fs');
var readableStream = fs.createReadStream('file.txt');
var data = '';
var chunk;

readableStream.on('readable', function(){
 while ((chunk = readableStream.read()) != null) {
 data += chunk;
 }
});

readableStream.on('end', function(){
 console.log(data);
});

The read () method reads data from the internal buffer and returns null when there is no data to read.
Therefore, in the while loop, we check whether read () returns null, and when it returns null, the loop is terminated.
Note that the readable event is triggered when we can read data from the stream.
Set code
By default, what you read from the stream is an Buffer object. If you want to read a string, this is not suitable for you. Therefore, you can set the encoding of the stream by calling Readable. setEncoding () as in the following example:

 var fs = require('fs');
var readableStream = fs.createReadStream('file.txt');
var data = '';

readableStream.setEncoding('utf8');

readableStream.on('data', function(chunk){
 data += chunk;
});

readableStream.on('end', function(){
 console.log(data);
});

In the above example, we set the encoding of the stream to utf8, the data will be parsed to utf8, and chunk in the callback function will be a string.
Pipeline (Piping)
Pipelines are a great mechanism for reading data from a data source and writing it to a destination without having to manage the state of the flow yourself. Let's look at the following example first:

 var fs = require('fs');
var readableStream = fs.createReadStream('file1.txt');
var writableStream = fs.createWriteStream('file2.txt');

readableStream.pipe(writableStream);

The above example uses the pipe () method to write the contents of file1 to file2. Because pipe () will help you manage the data flow, you don't need to worry about the speed of the data flow. This makes pipe () very simple and easy to use.
Note that pipe () returns the destination stream, so you can easily link multiple streams together!
Link (Chaining)
Suppose there is an archive file and you want to unzip it. There are many ways to accomplish this task. But the simplest way is to use pipes and links:

 var fs = require('fs');
var zlib = require('zlib');

fs.createReadStream('input.txt.gz')
 .pipe(zlib.createGunzip())
 .pipe(fs.createWriteStream('output.txt'));

First, we create a readable stream through input. txt. gz, and then let it stream the zlib. createGunzip () stream, which extracts the content. Finally, we add a writable stream to write the extracted contents to another file.
Other methods
Now that we've discussed some important concepts in readable streams, here are some more methods you need to know:
1. Readable. pause () This method pauses the flow of the stream. In other words, it will no longer trigger the data event.
2. Readable. resume () This method, contrary to the above, returns the suspended flow to flow.
3. Readable. unpipe () This method removes the destination. If a parameter is passed in, it causes the readable stream to stop at a particular destination; Otherwise, it removes all destinations.

Writable Stream (Writable Streams)
Writable streams allow you to write data to your destination. Like readable streams, these are also EventEmitter, and they also trigger different events. Let's look at the events and methods triggered in the writable stream.
Write stream
To write data to a writable stream, you need to call the write () method in the writable stream instance. Look at the following example:

 var fs = require('fs');
var readableStream = fs.createReadStream('file1.txt');
var writableStream = fs.createWriteStream('file2.txt');

readableStream.setEncoding('utf8');

readableStream.on('data', function(chunk){
 writableStream.write('chunk');
});

The above code is very simple; It simply reads the data from the input stream and writes it to the destination using write ().
This method returns 1 Boolean value to indicate whether the write was successful. If true is returned, it means that the write was successful, and you can continue to write more data. If it's false, that means something went wrong and you can't continue writing now. A writable stream fires an drain event to tell you that you can continue writing data.
After writing the data,
When you don't need to write data, you can call the end () method to tell the stream that you have finished writing. Assuming res is an HTTP response object, you usually send a response to the browser:
res.write('Some Data!!');
res.end();
When end () is called, all data is written and the stream fires an finish event. Note that after calling end (), you can no longer write data to the writable stream. For example, the following code will report an error:
res.write('Some Data!!');
res.end();
res.write('Trying to write again'); //Error !
Here are a few important events related to writable streams:
1. error triggers when a write or link error occurs
2. pipe This event is triggered when a readable stream is linked to a writable stream
3. unpipe is triggered when the readable stream calls unpipe