Tuesday, December 22, 2020

Node.js File System

 Node.js includes fs module to access physical file system. The fs module is responsible for all the asynchronous or synchronous file I/O operations.

Let's see some of the common I/O operation examples using fs module.

Reading File

Use fs.readFile() method to read the physical file asynchronously.

Signature:
fs.readFile(fileName [,options], callback)

Parameter Description:

  • filename: Full path and name of the file as a string.
  • options: The options parameter can be an object or string which can include encoding and flag. The default encoding is utf8 and default flag is "r".
  • callback: A function with two parameters err and fd. This will get called when readFile operation completes.

The following example demonstrates reading existing TestFile.txt asynchronously.

Example: Reading File
var fs = require('fs');

fs.readFile('TestFile.txt', function (err, data) {
                    if (err) throw err;

    console.log(data);
});

The above example reads TestFile.txt (on Windows) asynchronously and executes callback function when read operation completes. This read operation either throws an error or completes successfully. The err parameter contains error information if any. The data parameter contains the content of the specified file.

The following is a sample TextFile.txt file.

TextFile.txt
This is test file to test fs module of Node.js

Now, run the above example and see the result as shown below.

C:\> node server.js
This is test file to test fs module of Node.js

Use fs.readFileSync() method to read file synchronously as shown below.

Example: Reading File Synchronously
var fs = require('fs');

var data = fs.readFileSync('dummyfile.txt', 'utf8');
console.log(data);

Writing File

Use fs.writeFile() method to write data to a file. If file already exists then it overwrites the existing content otherwise it creates a new file and writes data into it.

Signature:
fs.writeFile(filename, data[, options], callback)

Parameter Description:

  • filename: Full path and name of the file as a string.
  • Data: The content to be written in a file.
  • options: The options parameter can be an object or string which can include encoding, mode and flag. The default encoding is utf8 and default flag is "r".
  • callback: A function with two parameters err and fd. This will get called when write operation completes.

The following example creates a new file called test.txt and writes "Hello World" into it asynchronously.

Example: Creating & Writing File
var fs = require('fs');

fs.writeFile('test.txt', 'Hello World!', function (err) { 
                        if (err)
        console.log(err);
                        else
        console.log('Write operation complete.');
});

In the same way, use fs.appendFile() method to append the content to an existing file.

Example: Append File Content
var fs = require('fs');

fs.appendFile('test.txt', 'Hello World!', function (err) { 
                        if (err)
        console.log(err);
                        else
        console.log('Append operation complete.');
});
ADVERTISEMENT

Open File

Alternatively, you can open a file for reading or writing using fs.open() method.

Signature:
fs.open(path, flags[, mode], callback)

Parameter Description:

  • path: Full path with name of the file as a string.
  • Flag: The flag to perform operation
  • Mode: The mode for read, write or readwrite. Defaults to 0666 readwrite.
  • callback: A function with two parameters err and fd. This will get called when file open operation completes.

Flags

The following table lists all the flags which can be used in read/write operation.

FlagDescription
rOpen file for reading. An exception occurs if the file does not exist.
r+Open file for reading and writing. An exception occurs if the file does not exist.
rsOpen file for reading in synchronous mode.
rs+Open file for reading and writing, telling the OS to open it synchronously. See notes for 'rs' about using this with caution.
wOpen file for writing. The file is created (if it does not exist) or truncated (if it exists).
wxLike 'w' but fails if path exists.
w+Open file for reading and writing. The file is created (if it does not exist) or truncated (if it exists).
wx+Like 'w+' but fails if path exists.
aOpen file for appending. The file is created if it does not exist.
axLike 'a' but fails if path exists.
a+Open file for reading and appending. The file is created if it does not exist.
ax+Like 'a+' but fails if path exists.

The following example opens an existing file and reads its content.

Example:File open and read
var fs = require('fs');

fs.open('TestFile.txt', 'r', function (err, fd) {
    
                            if (err) {
                            return console.error(err);
    }
    
                            var buffr = new Buffer(1024);
    
    fs.read(fd, buffr, 0, buffr.length, 0, function (err, bytes) {
       
                            if (err) throw err;
            
                            // Print only read bytes to avoid junk.
                            if (bytes > 0) {
            console.log(buffr.slice(0, bytes).toString());
        }
        
                            // Close the opened file.
        fs.close(fd, function (err) {
                            if (err) throw err;
        });
    });
});

Delete File

Use fs.unlink() method to delete an existing file.

Signature:
fs.unlink(path, callback);

The following example deletes an existing file.

Example:File Open and Read
var fs = require('fs');

fs.unlink('test.txt', function () {
    
    console.log('write operation complete.');

});

Important method of fs module

MethodDescription
fs.readFile(fileName [,options], callback)Reads existing file.
fs.writeFile(filename, data[, options], callback)Writes to the file. If file exists then overwrite the content otherwise creates new file.
fs.open(path, flags[, mode], callback)Opens file for reading or writing.
fs.rename(oldPath, newPath, callback)Renames an existing file.
fs.chown(path, uid, gid, callback)Asynchronous chown.
fs.stat(path, callback)Returns fs.stat object which includes important file statistics.
fs.link(srcpath, dstpath, callback)Links file asynchronously.
fs.symlink(destination, path[, type], callback)Symlink asynchronously.
fs.rmdir(path, callback)Renames an existing directory.
fs.mkdir(path[, mode], callback)Creates a new directory.
fs.readdir(path, callback)Reads the content of the specified directory.
fs.utimes(path, atime, mtime, callback)Changes the timestamp of the file.
fs.exists(path, callback)Determines whether the specified file exists or not.
fs.access(path[, mode], callback)Tests a user's permissions for the specified file.
fs.appendFile(file, data[, options], callback)Appends new content to the existing file.

Visit Node documentation for more information on fs module.

NPM - Node Package Manager

 Node Package Manager (NPM) is a command line tool that installs, updates or uninstalls Node.js packages in your application. It is also an online repository for open-source Node.js packages. The node community around the world creates useful modules and publishes them as packages in this repository.

It has now become a popular package manager for other open-source JavaScript frameworks like AngularJS, jQuery, Gulp, Bower etc.

Official website: https://www.npmjs.com

NPM is included with Node.js installation. After you install Node.js, verify NPM installation by writing the following command in terminal or command prompt.

C:\> npm -v
2.11.3

If you have an older version of NPM then you can update it to the latest version using the following command.

C:\> npm install npm -g

To access NPM help, write npm help in the command prompt or terminal window.

C:\> npm help

NPM performs the operation in two modes: global and local. In the global mode, NPM performs operations which affect all the Node.js applications on the computer whereas in the local mode, NPM performs operations for the particular local directory which affects an application in that directory only.

Install Package Locally

Use the following command to install any third party module in your local Node.js project folder.

C:\>npm install <package name>

For example, the following command will install ExpressJS into MyNodeProj folder.

C:\MyNodeProj> npm install express

All the modules installed using NPM are installed under node_modules folder. The above command will create ExpressJS folder under node_modules folder in the root folder of your project and install Express.js there.

Add Dependency into package.json

Use --save at the end of the install command to add dependency entry into package.json of your application.

For example, the following command will install ExpressJS in your application and also adds dependency entry into the package.json.

C:\MyNodeProj> npm install express --save

The package.json of NodejsConsoleApp project will look something like below.

package.json
{
  "name": "NodejsConsoleApp",
  "version": "0.0.0",
  "description": "NodejsConsoleApp",
  "main": "app.js",
  "author": {
    "name": "Dev",
    "email": ""
  },
  "dependencies": {
    "express": "^4.13.3"
  }
}

Install Package Globally

NPM can also install packages globally so that all the node.js application on that computer can import and use the installed packages. NPM installs global packages into /<User>/local/lib/node_modules folder.

Apply -g in the install command to install package globally. For example, the following command will install ExpressJS globally.

C:\MyNodeProj> npm install -g express

Update Package

To update the package installed locally in your Node.js project, navigate the command prompt or terminal window path to the project folder and write the following update command.

C:\MyNodeProj> npm update <package name>

The following command will update the existing ExpressJS module to the latest version.

C:\MyNodeProj> npm update express

Uninstall Packages

Use the following command to remove a local package from your project.

C:\>npm uninstall <package name>

The following command will uninstall ExpressJS from the application.

C:\MyNodeProj> npm uninstall express

Visit docs.npmjs.com for more information on Node Package Manager.

Node.js Module

 Module in Node.js is a simple or complex functionality organized in single or multiple JavaScript files which can be reused throughout the Node.js application.

Each module in Node.js has its own context, so it cannot interfere with other modules or pollute global scope. Also, each module can be placed in a separate .js file under a separate folder.

Node.js implements CommonJS modules standard. CommonJS is a group of volunteers who define JavaScript standards for web server, desktop, and console application.

Node.js Module Types

Node.js includes three types of modules:

  1. Core Modules
  2. Local Modules
  3. Third Party Modules

Node.js Core Modules

Node.js is a light weight framework. The core modules include bare minimum functionalities of Node.js. These core modules are compiled into its binary distribution and load automatically when Node.js process starts. However, you need to import the core module first in order to use it in your application.

The following table lists some of the important core modules in Node.js.

Core ModuleDescription
httphttp module includes classes, methods and events to create Node.js http server.
urlurl module includes methods for URL resolution and parsing.
querystringquerystring module includes methods to deal with query string.
pathpath module includes methods to deal with file paths.
fsfs module includes classes, methods, and events to work with file I/O.
utilutil module includes utility functions useful for programmers.

Loading Core Modules

In order to use Node.js core or NPM modules, you first need to import it using require() function as shown below.

var module = require('module_name');

As per above syntax, specify the module name in the require() function. The require() function will return an object, function, property or any other JavaScript type, depending on what the specified module returns.

The following example demonstrates how to use Node.js http module to create a web server.

Example: Load and Use Core http Module
var http = require('http');

var server = http.createServer(function(req, res){

  //write code here

});

server.listen(5000); 

In the above example, require() function returns an object because http module returns its functionality as an object, you can then use its properties and methods using dot notation e.g. http.createServer().

In this way, you can load and use Node.js core modules in your application. We will be using core modules throughout these tutorials.

Learn about local modules in the next section.

Node.js Basics

 Node.js supports JavaScript. So, JavaScript syntax on Node.js is similar to the browser's JavaScript syntax.

Visit JavaScript section to learn about JavaScript syntax in detail.

Primitive Types

Node.js includes following primitive types:

  • String
  • Number
  • Boolean
  • Undefined
  • Null
  • RegExp

Everything else is an object in Node.js.

Loose Typing

JavaScript in Node.js supports loose typing like the browser's JavaScript. Use var keyword to declare a variable of any type.

Object Literal

Object literal syntax is same as browser's JavaScript.

Example: Object
var obj = {
    authorName: 'Ryan Dahl',
    language: 'Node.js'
}

Functions

Functions are first class citizens in Node's JavaScript, similar to the browser's JavaScript. A function can have attributes and properties also. It can be treated like a class in JavaScript.

Example: Function
function Display(x) { 
    console.log(x);
}

Display(100);

Buffer

Node.js includes an additional data type called Buffer (not available in browser's JavaScript). Buffer is mainly used to store binary data, while reading from a file or receiving packets over the network.

process object

Each Node.js script runs in a process. It includes process object to get all the information about the current process of Node.js application.

The following example shows how to get process information in REPL using process object.

> process.execPath
'C:\\Program Files\\nodejs\\node.exe'
> process.pid
1652
> process.cwd()
'C:\\'

Defaults to local

Node's JavaScript is different from browser's JavaScript when it comes to global scope. In the browser's JavaScript, variables declared without var keyword become global. In Node.js, everything becomes local by default.

Access Global Scope

In a browser, global scope is the window object. In Node.js, global object represents the global scope.

To add something in global scope, you need to export it using export or module.export. The same way, import modules/object using require() function to access it from the global scope.

For example, to export an object in Node.js, use exports.name = object.

Example:
exports.log = {
    console: function(msg) {
        console.log(msg);
    },
    file: function(msg) {
        // log to file here
      }
}

Now, you can import log object using require() function and use it anywhere in your Node.js project.

Learn about modules in detail in the next section.

Kafka setup in window

Here's a step-by-step guide on how to do it : 1. Prerequisites : Before you begin, make sure you have the following prerequisites inst...