Node.js file operation details
- 2020-03-30 03:42:49
- OfStack
Node has a data flow API that handles files like a network stream, which is convenient to use, but it only allows sequential processing of files, not random reads and writes. Therefore, you need to use some lower-level file system operations.
This chapter covers the basics of file processing, including how to open a file, read a part of the file, write data, and close the file.
Many of the file apis for Node are almost a copy of the UNIX (POSIX) file apis, such as the way they use file descriptors, which, like UNIX, are an integer number in Node, representing the index of an entity in the process file descriptor table.
There are three special file descriptors -- 1,2, and 3. They represent standard input, standard output, and standard error file descriptors, respectively. Standard input, as its name implies, is a read-only stream that processes use to read data from the console or process channel. Standard output and standard error are file descriptors that only output data, and they are often used to output data to the console, other processes, or files. Standard error is responsible for the error message output, while standard output is responsible for the normal process output.
Once the process is started, you can use these file descriptors, which don't actually have a physical file. You can't write to and read from specific positions within the file, and You can only read and output in the same order as a network stream.
Normal files are not subject to this restriction. In Node, for example, you can create files that can only append data to the tail, and you can also create files that can read and write to random locations.
Almost all file-related operations involve processing file paths, and this chapter will cover these utility functions first, then dive into file read and write and data operations
Process file path
The file path is divided into relative path and absolute path. You can merge file paths, extract file name information, and even check if a file exists.
In the Node, can use a string to hold processing file path, but that will make complex problems, such as you want to connect different parts of the path, some part of the end with "/" no, and the path separator in the different operating systems may be different, so, when you connect them, the code can be very bothersome and trouble.
Fortunately, Node has a module called path that can help you standardize, connect, parse, convert from an absolute path to a relative path, extract information from the path, and detect the existence of files. In general, the path module is just string manipulation, and it doesn't go to the file system for validation (the path.exists function exception).
Path normalization
It is usually a good idea to standardize paths before they are stored or used. For example, file paths obtained by user input or configuration files, or paths joined by two or more paths, should generally be standardized. You can normalize a path with the normalize function of the path module, and it also handles ".." "/ /", ". ". Such as:
var path = require('path');
path.normalize('/foo/bar//baz/asdf/quux/..');
// => '/foo/bar/baz/asdf'
Connection path
Using the path.join() function, you can concatenate any number of path strings. You can simply pass all the path strings to the join() function:
var path = require('path');
path.join('/foo', 'bar', 'baz/asdf', 'quux', '..');
// => '/foo/bar/baz/asdf'
As you can see, path.join() automatically normalizes paths internally.
Parsing path
Use path.resolve() to resolve multiple paths to an absolute path. Unlike the arguments to the CD command, these paths can be files, and they don't have to actually exist -- the path.resolve() method doesn't access the underlying file system to determine if the path exists, it's just a bunch of string operations.
Such as:
var path = require('path');
path.resolve('/foo/bar', './baz');
// => /foo/bar/baz
path.resolve('/foo/bar', '/tmp/file/');
// => /tmp/file
If the resolution result is not an absolute path, path.resolve() appends the current working directory as the path to the resolution result, for example:
path.resolve('wwwroot', 'static_files/png/', '../gif/image.gif');
//If the current working directory is /home/myself/node, it returns
// => /home/myself/node/wwwroot/static_files/gif/image.gif'
Calculate the relative paths of two absolute paths
Path. Relative () tells you how to jump from one absolute address to another, for example:
var path = require('path');
path.relative('/data/orandea/test/aaa', '/data/orandea/impl/bbb');
// => ../../impl/bbb
Extract data from the path
Take the path "/foo/bar/myfile.txt" as an example. If you want to get all the contents of the parent directory (/foo/bar), or to read other files in the same directory, you must use path.dirname(filePath) to get the directory part of the filePath, for example:
var path = require('path');
path.dirname('/foo/bar/baz/asdf/quux.txt');
// => /foo/bar/baz/asdf
Or, if you want to get the file name from the file path, which is the last part of the file path, you can use the path.basename function:
var path = require('path');
path.basename('/foo/bar/baz/asdf/quux.html')
// => quux.html
The file path may also contain the file extension, usually the string after the last ". "character in the file name.
Path. Basename can also accept an extension string as a second parameter, so the returned filename will automatically drop the extension and just return the name part of the file:
var path = require('path');
path.basename('/foo/bar/baz/asdf/quux.html', '.html');
// => quux
To do this, you first need to know the file's extension, you can use path.extname() to get the extension:
var path = require('path');
path.extname('/a/b/index.html');
// => '.html'
path.extname('/a/b.c/index');
// => ''
path.extname('/a/b.c/.');
// => ''
path.extname('/a/b.c/d.');
// => '.'
Check if the path exists
So far, the path-handling operations covered above are not related to the underlying file system, just string operations. However, there are times when you need to determine if a file path exists. For example, you sometimes need to determine if a file or directory exists and create it if it doesn't. Use path.exsits() :
var path = require('path');
path.exists('/etc/passwd', function(exists) {
console.log('exists:', exists);
// => true
});
path.exists('/does_not_exist', function(exists) {
console.log('exists:', exists);
// => false
});
Note: starting with Node0.8, exists has been moved from the path module to the fs module and changed to fs. Exists, nothing has changed except the namespace:
var fs = require('fs');
fs.exists('/does_not_exist', function(exists) {
console.log('exists:', exists);
// => false
});
Path. Exists () is an I/O operation because it is asynchronous and therefore requires a callback function that is called when the I/O operation returns and the result is passed to it. You can also use a synchronized version of path.existssync (), which does exactly the same thing, except that it doesn't call callbacks and instead returns the result:
var path = require('path');
path.existsSync('/etc/passwd');
// => true
Introduction to fs module
The fs module contains all the functions related to file query and processing, with which you can query file information, read and write files, and close files. Import fs module like this:
var fs = require( ' fs')
Query file information
Sometimes you may need to know the size of the file, the date of creation, or the permissions of the file, etc. You can use the fs.stath function to query the meta information of the file or directory:
var fs = require('fs');
fs.stat('/etc/passwd', function(err, stats) {
if (err) { throw err;}
console.log(stats);
});
This code snippet will have output similar to the following
{ dev: 234881026,
ino: 95028917,
mode: 33188,
nlink: 1,
uid: 0,
gid: 0,
rdev: 0,
size: 5086,
blksize: 4096,
blocks: 0,
atime: Fri, 18 Nov 2011 22:44:47 GMT,
mtime: Thu, 08 Sep 2011 23:50:04 GMT,
ctime: Thu, 08 Sep 2011 23:50:04 GMT }
The 1.fs.stat() call passes an instance of the stats class as an argument to its callback function, and the stats instance can be used as follows:
Stats. IsFile () -- returns true or false if it is a standard file, not a directory, socket, symbolic link, or device
Stats. IsDiretory () -- returns a tue if it is a directory, or false otherwise
Stats.isblockdevice () -- returns true if it is a block device, which is typically in the /dev directory on most UNIX systems
Stats. IsChracterDevice () -- returns true if it is a character device
6. Stats. IsSymbolickLink () -- returns true if it is a file link
Stats. IsFifo () -- returns true if it is a FIFO(a special type of UNIX named pipe)
Stats. IsSocket () -- if it's a UNIX socket (TODO: googe it)
Open the file
Before you can read or process the file, you must first open the file using the fs.open function. Then the callback function you provide will be called and the descriptor of the file will be obtained.
var fs = require('fs');
fs.open('/path/to/file', 'r', function(err, fd) {
// got fd file descriptor
});
The first parameter of fs.open is the file path, and the second parameter is some tags that indicate the mode in which the file is opened. These tags can be r, r+, w, w+, a, or a+. Here is a description of these tags (from the fopen page of the UNIX documentation)
1. R -- open the file read-only, with the initial location of the data stream at the beginning of the file
2. R + -- open the file in read-write mode, with the initial location of the data stream at the beginning of the file
3. W -- if the file exists, clear the length of the file by 0, meaning the contents of the file will be lost. If it does not exist, try to create it. The initial location of the data flow is at the beginning of the file
4. W + -- open the file in read-write mode, try to create the file if it doesn't exist, and clear the file length to 0 if it does, meaning the file contents will be lost. The initial location of the data flow is at the beginning of the file
5. A -- open the file as a write-only, and if the file doesn't exist, try to create it. The data stream starts at the end of the file, and each subsequent write appends the data to the end of the file.
6. A + -- open the file in read-write mode, and if the file does not exist, try to create it. The data stream starts at the end of the file, and each subsequent write appends the data to the end of the file.
Read the file
Once you open the file, you can start reading the contents of the file, but before you can, you have to create a buffer to hold the data. This buffer object will be passed to the fs.read function as an argument and filled with data by fs.read.
var fs = require('fs');
fs.open('./my_file.txt', 'r', function opened(err, fd) {
if (err) { throw err }
var readBuffer = new Buffer(1024),
bufferOffset = 0,
bufferLength = readBuffer.length,
filePosition = 100;
fs.read(fd,
readBuffer,
bufferOffset,
bufferLength,
filePosition,
function read(err, readBytes) {
if (err) { throw err; }
console.log('just read ' + readBytes + ' bytes');
if (readBytes > 0) {
console.log(readBuffer.slice(0, readBytes));
}
});
});
The above code attempts to open a file, and when it is successfully opened (the opened function is called), the request begins to read the subsequent 1024 bytes from the 100th byte of the file stream (line 11).
The last argument to fs.read() is a callback function (line 16), which is called when:
1. An error occurred
2. The data was read successfully
3. No data to read
If an error occurs, the first parameter (err) provides an object with the error message for the callback function, otherwise the parameter is null. If the data is read successfully, the second parameter (readBytes) indicates the size of the data to be read into the buffer, and if the value is 0, the end of the file is reached.
Note: once the buffer object is passed to fs.open(), control of the buffer object is transferred to the read command, and control of the buffer object is returned to you only when the callback function is called. So don't read or write or let other function calls use this buffer object until then; Otherwise, you may read incomplete data, or worse, you may write data to the buffer object concurrently.
Write files
By passing a buffer containing data to fs.write(), write data to and from an open file:
var fs = require('fs');
fs.open('./my_file.txt', 'a', function opened(err, fd) {
if (err) { throw err; }
var writeBuffer = new Buffer('writing this string'),
bufferPosition = 0,
bufferLength = writeBuffer.length, filePosition = null;
fs.write( fd,
writeBuffer,
bufferPosition,
bufferLength,
filePosition,
function wrote(err, written) {
if (err) { throw err; }
console.log('wrote ' + written + ' bytes');
});
});
In this example, line 2 tries to open a file in append mode (a), and then line 7 writes data to the file. The buffer object needs to come with several messages as parameters:
1. Buffer data
2. Where does the pending data start in the buffer
3. Length of data to be written
4. Where the data is written to the file
The callback function called after the operation is completed
In this example, the filePostion parameter is null, which means that the write function will write the data to the current position of the file pointer, because the file is opened in append mode, so the file pointer is at the end of the file.
As with the read operation, never use an incoming buffer object during fs.write; it gains control of that buffer object once fs.write has started. You can't reuse a callback until it is called.
Close the file
You may have noticed that so far, none of the examples in this chapter have code to close files. Because they are small, simple examples that are used only once, the operating system makes sure to close all the files when the Node process ends.
However, in a real application, once you open a file you want to make sure you eventually close it. To do this, you need to keep track of all those open file descriptors and then call fs.close(fd[,callback]) when they are no longer used to eventually close them. It's easy to miss a file descriptor if you're not careful. The following example provides a function called openAndWriteToSystemLog that shows you how to close files carefully:
var fs = require('fs');
function openAndWriteToSystemLog(writeBuffer, callback){
fs.open('./my_file', 'a', function opened(err, fd) {
if (err) { return callback(err); }
function notifyError(err) {
fs.close(fd, function() {
callback(err);
});
}
var bufferOffset = 0,
bufferLength = writeBuffer.length,
filePosition = null;
fs.write( fd, writeBuffer, bufferOffset, bufferLength, filePosition,
function wrote(err, written) {
if (err) { return notifyError(err); }
fs.close(fd, function() {
callback(err);
});
}
);
});
}
openAndWriteToSystemLog(
new Buffer('writing this string'),
function done(err) {
if (err) {
console.log("error while opening and writing:", err.message);
return;
}
console.log('All done with no errors');
}
);
Here, a function called openAndWriteToSystemLog is provided, which accepts a buffer object containing data to be written, and a callback function that is called after an operation is completed or if an error occurs, the first argument to the callback function contains the error object.
Notice the internal function notifyError, which closes the file and reports the error.
Note: at this point, you know how to use the underlying atomic operations to open, read, write, and close files. However, Node also has a set of more advanced constructors that allow you to work with files in a simpler way.
For example, if you want two or more write operations to append data to a file in a safe way, you can use WriteStream.
Also, if you want to read an area of a file, consider using ReadStream. These two use cases are described in chapter 9, "read data, write streams."
summary
Most of the time when you are working with files, you need to process and extract file path information. By using the path module, you can connect paths, standardize paths, calculate path differences, and convert relative paths to absolute paths. You can extract the specified file path's extension, file name, directory, and other path components.
Node provides an underlying API in the fs module to access the file system, and the underlying API USES file descriptors to manipulate files. You can open a file with fs.open, write it with fs.write, read it with fs.read, and close it with fs.close.
When an error occurs, you should always use the correct error-handling logic to close the file -- to ensure that the open file descriptors are closed before the call returns.