Node js Architecture

Introduction

Node.js is completely free and open source. It is also used and supported by a large number of developers all around the world. Node.js can be called the combination of Chrome’s V8 Js engine, event loop, and low-level I/O API. The below diagram shows the architecture of Node.js:

Node.js has its core part written in C and C++. Node js is based on a single-threaded event loop architecture which allows Node to handle multiple client requests. Node js uses the concept of an asynchronous model and non-blocking I/O. We will look at these terms in detail.

Node.js is one of the best options for other server-side platforms because of the architectural pattern it follows.

Single-Threaded Event Loop Architecture in Node.js

Node.js is based on the single threaded event driven architecture. It has a Non-blocking I/O model. Let us look at these concepts in detail :

Node.js is Single Threaded Event Loop

The Event loop in Node.js is single-threaded. All requests may have two parts - synchronous and asynchronous. The event loop or the main thread takes up the synchronous part and assigns the asynchronous part to a background thread to execute. The main thread then takes up the synchronous part of other requests.

Example:

Node has Non Blocking I/O Model

The Main thread in Node.js allocates the time-consuming asynchronous operation to a background thread. It does not wait for the background thread to complete that operation; it takes up the next operation in the event queue.

The main thread only handles the synchronous part of every request. After the background thread completes the async task, it notifies the main thread to take up the operation for further execution of the callback code. Hence the main thread is not blocked while I/O is being performed by the async operation.

Example of the filesystem module. Following is an example of a blocking operation:

The asynchronous version of the above code that shows non-blocking:

Event-Based, Instead of Waiting for I/O Operation

The thread running in the background performing some asynchronous operation notifies the main thread using Events. After the async task is completed, the associated callback function is needed to be called. If the main thread is busy executing some other request, then whenever it becomes idle, the main thread reacts to the event notification of the background thread. It runs the callback code and sends back the response to the client.

In the above code, after the file is read the associated callback function is executed.

Components of the Node.js Architecture

Let us see all the components one by one :

• Requests: Requests are sent by clients to fetch some server resources or by the server itself to some other server to fetch resources from it. Hence the resource can be of two types: incoming and outgoing.
• Node js Server: It is the server-side or the backend platform that receives requests from users at various endpoints defined. All the requests are processed by the server, and the response generated is sent back to the client. The server constantly listens on a specified port in the system. The clients send requests by specifying the target IP and port of the server.
• Event loop: It is an infinite loop that has six phases. All these six phases are repeated until there is no code left to execute. In these six phases, the event loop receives requests, processes them, and returns the response of these requests to the clients. The six phases of the event loop are:
  1. Timers
  2. I/O Callbacks
  3. Waiting / Preparation
  4. I/O Polling
  5. setImmediate() callbacks
  6. Close events

• Event Queue: All the requests sent by the clients are stored in the event queue. They are then passed one by one to the event loop. The callbacks of the operations running on other threads are also added to it so that the main thread takes them up.
• Thread pool: Other than the main thread, there are other threads managed in a thread pool. All the asynchronous processes and non-blocking I/O are attached to one of these threads as they continue executing in the background.
• External resources: These are the resources that the server has to fetch to fulfill client requests. They are needed to deal with the blocking requests. E.g., computation, data storage, etc.

Workflow of Nodejs Server

The workflow of a web server created with Node.js involves all the components discussed in the above section. The entire architectural work has been illustrated in the below diagram.

• Clients send requests to the web server. These requests can be either blocking (complex) or non-blocking (simple). The purpose of the requests may be to query for data, delete data or update data.
• The requests are retrieved and added to the event queue.
• The requests are then passed from the event queue to the event loop one by one.
• The simple requests are handled by the main thread, and the response is sent back to the client.
• A complex request is assigned to a thread from the thread pool.

Best Practices for Node.js Application Architecture

Incorporate the Publisher-Subscriber Model

The pub/sub model is a messaging pattern in which the components publish messages and others subscribe to them. This is done to notify others and send data to them. Hence there are two components involved in this communication system - the publishers and the subscribers.

Messages are sent through specified channels by the publishers without any knowledge of the subscribers or the receiving end. The subscribers on the other end of these channels listen to these messages without the knowledge of the publishers. Hence multiple nodes can be connected using this data-sharing model. All the subscribers can listen to one action and respond to it.

Let us now understand the concept with the help of some examples. If we have to send notifications about some event to a large number of users, then it may require sending messages to the clients in a rapid way. This is a big challenge. It becomes bigger when the number of users gets scaled up. When the users are not active, there is a chance that users will miss these messages and not respond to them. Therefore, it falls on the system to make sure the client receives the message when it is active.

The pub/sub model allows the decoupling of the components, and each component is bound to rely on a message broker. The publishers send messages, and the clients subscribe to them to receive messages from the broker. The brokers are implemented to save the message and deliver the message when the client comes online. Developers are hence not required to worry about the interface and structure of other components. This also allows the scalability of the system.

Adopt a Layered Approach

The use of express.js allows the easy distribution of the logic of the application into various categories. By dividing the concerns, the code quality improves and debugging becomes a lot easier. The setup separates the logic of the application from the API routes so that the background processes don't become complex. The code is divided into three categories - business logic, database, and API routes.

Controller layer: This is where the API routes are defined. The request is destructured here, and based on the information in the request, some processing is done. The processing done is collected and passed on to the service layer.

Service layer: The business logic related to the application is defined here. The classes and functions used are included in this layer. The SOLID principles of Object-oriented programming are followed. The processing logic for various routes is also in this layer.

Data access layer: This layer takes care of the database handling. All the reading, writing, or any other manipulation of the database is handled by this layer. The SQL queries, connections to the database, document models, and other related code is defined here.

Use Dependency Injection

Dependency injection is a design pattern for software in which the dependencies of the software are passed in as parameters rather than including them or creating the dependencies inside the software. This technique improves the flexibility, independence, scalability, and reusability of the modules. The testing also becomes easy.

Let’s take an example to understand this concept :

In the above code example, we are focused only on managing the posts rather than how the posts are stored. The dependency for fetching posts is passed in the PostManager, and it is used without any concern about the dependency’s internal working. The PostManager class only needs to know how the postStore is to be used.

The dependency injection improves the scalability and understandability of code. The code is also easy to test.

Utilize Third-party Solutions

Node.js is blessed with a large community of developers supporting it. NPM, the Node package manager, is a package manager that can be used to install all the third-party modules in Node.js easily. NPM has a lot of well-documented and maintained frameworks. Making use of all these makes development very easy, and developers can focus on the logic part more.

Some other Node.js libraries that can also help are:

• Moment (date and time)
• Nodemon (automatically restarts the app when there’s a code update
• Agenda (job scheduling)
• Winston (logging)
• Grunt (automatic task runner)

There are several third-party modules available. But they should be used smartly. Overuse or relying on these modules is not good as it may affect the safety of the application. Also, they may introduce a large number of dependencies in the project.

Apply a Uniform Folder Structure

We have already discussed the layered structure. The division of code into various modules helps in making debugging and testing easy. It also promotes reusability.

Here is a basic folder structure you should follow while setting up a new application in Node.js:

The controller layer is the API directory, the service layer is the Services directory, and the data access layer is the Models directory. /config stores the environment variables, while /scripts store's workflow automation scripts. The /test directory contains the test cases, and /or subscribers store the event handlers in the pub/sub pattern.

Use Linters, Formatters, Style Guides, and Comments for Clean Coding

Linting and formatting: Static code analyzers that check the code for bugs, errors, and other wrong constructs are called linters. They help in the identification of bugs and other harmful patterns in our code. Some examples are Jslint and Eslint. Formatters on the other hand ensure that a consistent style is followed in the project. A prettier code formatter is an example of a formatted one. Linters and formatters are now available as plugins in many IDEs.

Style guides: Style guides help in following the naming conventions and other coding standards used by top developers. Some of the examples are style guides by Google and Airbnb.

Adding comments: To make code more readable and understandable to others we must use comments at every part of the project. They tell others what logic you have used in the current part of the code. Comments are also a good way to document the details like author, functionality, and other purposes.

Rectify Errors with Unit Testing, Logging, and Error-Handling

Unit testing: It is done to check the accuracy and correctness of one unit or part of the code. This helps reduce the debugging time and cost. It helps in verifying that the individual units are performing their tasks correctly or not. Jest, Mocha, and Jasmine are some frameworks for this purpose.

Logging and error-handling: Error is the problem or fault occurring in the program. It is shown with the error message and possible fixation. Many programming languages have built-in log generators. These logging systems are essential at every stage of the development process.

In Node.js, the most common way of logging is the use of the console.log() function that prints the information in the console.

There are three important streams checked in the logging process:

• Stdin – deals with input to the process (for example – keyboard or mouse)
• Stdout – deals with the output to the console or a separate file
• Stderr – deals specifically with error messages and warnings.

Handling errors is also an important part of the development of Node.js applications. Issues like callback hell must be addressed properly. A centralized error-handling component should be made to avoid duplication errors. The component should send messages to the admin, handle monitoring events and log everything. The use of try-catch should be done wherever necessary.

Using Config File and Environment Variables

When the application gets scaled, the need for global variables that can be accessed by every module arises. We can separate all the global variables in one file in the config folder. All the environment variables can be saved in a .env file.

The API keys, database passwords, and other such information can be stored this way. It gets saved as a .env file that has all the environment variables.

The dotenv package can be used to import all these environment variables. If any changes are to be made they can be done in one place and it will be reflected in the application.

Employ Gzip Compression

Whenever we have to transfer files, large files may create a problem. Hence using a file compression technique can help. Gzip is a lossless compression mechanism that can compress files so that they can be transferred quickly. You can compress the multimedia files that are served on the web pages to reduce the load and make processing faster.

Express.js helps in the easy implementation of compression techniques. In the documentation of Express.js, it is recommended to use Gzip compression.