Node.js Deep Dive: Build Scalable Backends with TypeScript

Node.js is an open-source, cross-platform JavaScript runtime environment built on Chrome’s V8 JavaScript engine. It allows developers to run JavaScript on the server side, enabling full-stack JavaScript development.

Key Features:

• Asynchronous and Event-Driven: Non-blocking I/O operations.
• Fast Execution: Built on V8, compiles JavaScript to native machine code.
• Single-Threaded but Highly Scalable: Uses event loop for concurrency.
• NPM (Node Package Manager): Largest ecosystem of open-source libraries.
• Cross-Platform: Runs on Windows, macOS, Linux.

Why Node.js?

• Full-stack JavaScript (reuse skills between frontend and backend).
• High performance for I/O-intensive applications (APIs, real-time apps).
• Large community and rich ecosystem.
• Ideal for microservices and serverless architectures.

#2. Installation and Setup

#Installing Node.js

Visit nodejs.org and download the LTS version.

Verify installation:

node --version
npm --version

#Setting Up a TypeScript Project

Initialize a new Node.js project with TypeScript:

mkdir my-node-app
cd my-node-app
npm init -y
npm install -D typescript @types/node ts-node nodemon
npx tsc --init

Configure tsconfig.json:

{
  "compilerOptions": {
    "target": "ES2020",
    "module": "commonjs",
    "outDir": "./dist",
    "rootDir": "./src",
    "strict": true,
    "esModuleInterop": true,
    "skipLibCheck": true
  },
  "include": ["src/**/*"],
  "exclude": ["node_modules"]
}

Update package.json scripts:

"scripts": {
  "build": "tsc",
  "start": "node dist/index.js",
  "dev": "nodemon src/index.ts"
}

Install nodemon and ts-node for development:

npm install -D nodemon ts-node

#3. Node.js Fundamentals

#What is Node.js?

Node.js is a runtime that uses an event-driven, non-blocking I/O model, making it lightweight and efficient for data-intensive real-time applications.

#Event-Driven Architecture

Node.js uses an event-driven architecture where events are emitted and handled by event listeners.

import { EventEmitter } from 'events';

const emitter = new EventEmitter();

emitter.on('greet', (name: string) => {
  console.log(`Hello, ${name}!`);
});

emitter.emit('greet', 'Alice');

#The Event Loop

The event loop allows Node.js to perform non-blocking I/O operations despite being single-threaded. It offloads operations to the system kernel whenever possible.

#Node.js Global Objects

• global: Global namespace (similar to window in browsers).
• __dirname: Current directory path.
• __filename: Current file path.
• process: Information about the current process.
• Buffer: Handle binary data.
• setTimeout, setInterval, setImmediate.

console.log(__dirname);
console.log(__filename);
console.log(process.pid);

#4. Core Modules

#File System (fs)

import fs from 'fs/promises';

async function readFileExample() {
  try {
    const data = await fs.readFile('file.txt', 'utf-8');
    console.log(data);
  } catch (err) {
    console.error(err);
  }
}

async function writeFileExample() {
  await fs.writeFile('output.txt', 'Hello, World!');
}

#HTTP Module

Create a basic HTTP server:

import http from 'http';

const server = http.createServer((req, res) => {
  res.writeHead(200, { 'Content-Type': 'text/plain' });
  res.end('Hello, World!\n');
});

server.listen(3000, () => {
  console.log('Server running at http://localhost:3000/');
});

#Path Module

import path from 'path';

const filePath = path.join(__dirname, 'files', 'data.json');
const ext = path.extname(filePath);
console.log(ext); // .json

#OS Module

import os from 'os';

console.log(os.platform()); // 'darwin', 'win32', etc.
console.log(os.cpus().length); // Number of CPU cores

#Events Module

Already covered; used extensively in Node.js core.

#Streams

Streams are used to handle reading/writing data sequentially.

import fs from 'fs';

const readStream = fs.createReadStream('largefile.txt', 'utf-8');
const writeStream = fs.createWriteStream('output.txt');

readStream.on('data', (chunk) => {
  writeStream.write(chunk);
});

readStream.on('end', () => {
  writeStream.end();
  console.log('Finished copying');
});

#Buffers

Buffers handle binary data.

const buf = Buffer.from('Hello', 'utf-8');
console.log(buf); // <Buffer 48 65 6c 6c 6f>
console.log(buf.toString()); // Hello

#5. NPM and Package Management

#What is NPM?

NPM is the default package manager for Node.js. It installs, manages, and shares reusable code.

#Package.json

The package.json file contains metadata and dependencies.

{
  "name": "my-app",
  "version": "1.0.0",
  "description": "",
  "main": "dist/index.js",
  "scripts": {
    "start": "node dist/index.js",
    "dev": "nodemon src/index.ts"
  },
  "dependencies": {
    "express": "^4.18.2"
  },
  "devDependencies": {
    "@types/express": "^4.17.17",
    "typescript": "^5.0.0"
  }
}

#Installing Packages

npm install express          # production dependency
npm install -D typescript    # dev dependency
npm install -g nodemon       # global install

#Semantic Versioning

• ^1.2.3 – Compatible with minor releases (1.x.x)
• ~1.2.3 – Compatible with patch releases (1.2.x)
• 1.2.3 – Exact version

#Creating and Publishing Packages

1. Create a package with npm init.
2. Write code.
3. Add a main entry point.
4. Publish with npm publish.

#6. Asynchronous Programming

#Callbacks

Traditional async handling.

import fs from 'fs';

fs.readFile('file.txt', 'utf-8', (err, data) => {
  if (err) {
    console.error(err);
    return;
  }
  console.log(data);
});

#Promises

Modern approach.

import fs from 'fs/promises';

fs.readFile('file.txt', 'utf-8')
  .then((data) => console.log(data))
  .catch((err) => console.error(err));

#Async/Await

Syntactic sugar over promises.

async function readFile() {
  try {
    const data = await fs.readFile('file.txt', 'utf-8');
    console.log(data);
  } catch (err) {
    console.error(err);
  }
}

#Error Handling in Async Code

Always use try/catch with async/await or .catch() with promises.

#7. Building Web Servers with Node.js

#Raw HTTP Server

Already shown in HTTP module.

#Express.js Framework

Express is the most popular web framework for Node.js.

Installation:

npm install express
npm install -D @types/express

Basic Express server:

import express, { Request, Response } from 'express';

const app = express();
const port = 3000;

app.get('/', (req: Request, res: Response) => {
  res.send('Hello, World!');
});

app.listen(port, () => {
  console.log(`Server running at http://localhost:${port}`);
});

#Routing

app.get('/users', (req, res) => {
  /* ... */
});
app.post('/users', (req, res) => {
  /* ... */
});
app.put('/users/:id', (req, res) => {
  /* ... */
});
app.delete('/users/:id', (req, res) => {
  /* ... */
});

#Middleware

Functions that execute during request-response cycle.

app.use(express.json()); // built-in middleware

// custom middleware
app.use((req, res, next) => {
  console.log(`${req.method} ${req.url}`);
  next();
});

#Request and Response Objects

• req.params, req.query, req.body
• res.send(), res.json(), res.status()

#Serving Static Files

app.use(express.static('public'));

#Template Engines (EJS, Pug)

npm install ejs

app.set('view engine', 'ejs');
app.get('/profile', (req, res) => {
  res.render('profile', { name: 'John' });
});

#8. Working with Databases

#MongoDB with Mongoose

npm install mongoose
npm install -D @types/mongoose

import mongoose from 'mongoose';

mongoose.connect('mongodb://localhost:27017/mydb');

const userSchema = new mongoose.Schema({
  name: { type: String, required: true },
  email: { type: String, required: true, unique: true },
});

const User = mongoose.model('User', userSchema);

// CRUD
async function createUser() {
  const user = new User({ name: 'Alice', email: 'alice@example.com' });
  await user.save();
}

#SQL Databases (PostgreSQL, MySQL) with Sequelize/Knex

npm install sequelize pg pg-hstore
npm install -D @types/sequelize

import { Sequelize, DataTypes } from 'sequelize';

const sequelize = new Sequelize('database', 'username', 'password', {
  host: 'localhost',
  dialect: 'postgres',
});

const User = sequelize.define('User', {
  name: { type: DataTypes.STRING, allowNull: false },
  email: { type: DataTypes.STRING, allowNull: false, unique: true },
});

await sequelize.sync();

#Connection Pooling

Both Mongoose and Sequelize handle connection pooling automatically.

#CRUD Operations

Basic operations: create, read, update, delete.

#9. RESTful API Development

#API Design Principles

• Use resource-based URLs (/users, /users/:id)
• HTTP methods (GET, POST, PUT, PATCH, DELETE)
• Stateless
• Use proper status codes

#HTTP Methods and Status Codes

#Request Validation

Use libraries like zod or joi.

npm install zod

import { z } from 'zod';

const userSchema = z.object({
  name: z.string().min(2),
  email: z.string().email(),
});

app.post('/users', (req, res) => {
  try {
    const validated = userSchema.parse(req.body);
    // save to db
    res.status(201).json(validated);
  } catch (err) {
    res.status(400).json({ error: err.errors });
  }
});

#Error Handling

Create a centralized error handler.

#Authentication & Authorization (JWT, OAuth)

npm install jsonwebtoken bcrypt
npm install -D @types/jsonwebtoken @types/bcrypt

import jwt from 'jsonwebtoken';
import bcrypt from 'bcrypt';

const secret = process.env.JWT_SECRET!;

// hash password
const hash = await bcrypt.hash(password, 10);

// compare password
const match = await bcrypt.compare(inputPassword, hash);

// generate token
const token = jwt.sign({ userId: user.id }, secret, { expiresIn: '1h' });

#API Documentation (Swagger/OpenAPI)

npm install swagger-jsdoc swagger-ui-express
npm install -D @types/swagger-jsdoc @types/swagger-ui-express

import swaggerJsdoc from 'swagger-jsdoc';
import swaggerUi from 'swagger-ui-express';

const options = {
  definition: {
    openapi: '3.0.0',
    info: { title: 'My API', version: '1.0.0' },
  },
  apis: ['./src/routes/*.ts'],
};

const specs = swaggerJsdoc(options);
app.use('/api-docs', swaggerUi.serve, swaggerUi.setup(specs));

#10. Middleware and Advanced Express

#Custom Middleware

function logger(req: Request, res: Response, next: NextFunction) {
  console.log(`${new Date().toISOString()} - ${req.method} ${req.url}`);
  next();
}

app.use(logger);

#Error Handling Middleware

app.use((err: Error, req: Request, res: Response, next: NextFunction) => {
  console.error(err.stack);
  res.status(500).json({ error: 'Something went wrong!' });
});

#Third-Party Middleware (Helmet, CORS, Morgan)

npm install helmet cors morgan
npm install -D @types/cors @types/morgan

import helmet from 'helmet';
import cors from 'cors';
import morgan from 'morgan';

app.use(helmet());
app.use(cors());
app.use(morgan('combined'));

#File Uploads (Multer)

npm install multer
npm install -D @types/multer

import multer from 'multer';

const upload = multer({ dest: 'uploads/' });

app.post('/upload', upload.single('file'), (req, res) => {
  res.json({ file: req.file });
});

#11. Real-Time Applications with WebSockets

#Socket.io

npm install socket.io

import { Server } from 'socket.io';
import http from 'http';

const server = http.createServer(app);
const io = new Server(server);

io.on('connection', (socket) => {
  console.log('a user connected');
  socket.on('chat message', (msg) => {
    io.emit('chat message', msg);
  });
  socket.on('disconnect', () => {
    console.log('user disconnected');
  });
});

server.listen(3000, () => {
  console.log('listening on *:3000');
});

#Building a Chat Application

See section 20 for a complete real-world example.

#12. Testing Node.js Applications

#Unit Testing with Jest/Mocha

npm install -D jest @types/jest ts-jest supertest @types/supertest
npx ts-jest config:init

// sum.ts
export const sum = (a: number, b: number) => a + b;

// sum.test.ts
import { sum } from './sum';

test('adds 1 + 2 to equal 3', () => {
  expect(sum(1, 2)).toBe(3);
});

#Integration Testing

Test API endpoints using Supertest.

import request from 'supertest';
import app from '../app';

describe('GET /', () => {
  it('responds with Hello World', async () => {
    const response = await request(app).get('/');
    expect(response.status).toBe(200);
    expect(response.text).toBe('Hello World');
  });
});

#Code Coverage

jest --coverage

#13. Error Handling and Debugging

#Error Types

• Operational errors (e.g., request timeout, database connection failure)
• Programmer errors (bugs)

#Try-Catch and Promises

Always catch promise rejections.

#Uncaught Exceptions and Unhandled Rejections

process.on('uncaughtException', (err) => {
  console.error('Uncaught Exception:', err);
  process.exit(1);
});

process.on('unhandledRejection', (reason, promise) => {
  console.error('Unhandled Rejection at:', promise, 'reason:', reason);
});

#Debugging with Node Inspector

node --inspect dist/index.js

Then open chrome://inspect in Chrome.

#Logging (Winston, Morgan)

npm install winston

import winston from 'winston';

const logger = winston.createLogger({
  level: 'info',
  format: winston.format.json(),
  transports: [
    new winston.transports.File({ filename: 'error.log', level: 'error' }),
    new winston.transports.File({ filename: 'combined.log' }),
  ],
});

if (process.env.NODE_ENV !== 'production') {
  logger.add(
    new winston.transports.Console({
      format: winston.format.simple(),
    })
  );
}

logger.info('Server started');

#14. Security Best Practices

#Helmet.js

Sets secure HTTP headers (already covered).

#CORS Configuration

app.use(cors({ origin: 'https://example.com' }));

#Rate Limiting

npm install express-rate-limit
npm install -D @types/express-rate-limit

import rateLimit from 'express-rate-limit';

const limiter = rateLimit({
  windowMs: 15 * 60 * 1000, // 15 minutes
  max: 100, // limit each IP to 100 requests per windowMs
});

app.use(limiter);

#Data Sanitization

Use libraries like express-mongo-sanitize, xss-clean.

#Environment Variables

Use dotenv:

npm install dotenv

import dotenv from 'dotenv';
dotenv.config();

const port = process.env.PORT || 3000;

#SQL Injection Prevention

Use ORM/ODM (Mongoose, Sequelize) which sanitize inputs.

#XSS and CSRF Protection

Use helmet and CSRF tokens with libraries like csurf.

#15. Performance and Optimization

#Clustering and Load Balancing

import cluster from 'cluster';
import os from 'os';

if (cluster.isMaster) {
  const numCPUs = os.cpus().length;
  for (let i = 0; i < numCPUs; i++) {
    cluster.fork();
  }
  cluster.on('exit', (worker) => {
    console.log(`worker ${worker.process.pid} died`);
    cluster.fork();
  });
} else {
  // workers share the server
  app.listen(3000);
}

#Using PM2 for Process Management

npm install -g pm2
pm2 start dist/index.js --name my-app -i max
pm2 monit

#Caching with Redis

npm install redis

import { createClient } from 'redis';

const client = createClient();
await client.connect();

await client.set('key', 'value');
const value = await client.get('key');

#Compression

npm install compression
npm install -D @types/compression

import compression from 'compression';
app.use(compression());

#Profiling and Benchmarking

Use node --prof, clinic, or autocannon.

#16. Deployment and DevOps

#Environment Configuration

Use .env files and environment-specific configs.

#Deploying to Heroku

heroku create my-app
git push heroku main
heroku open

#Deploying to AWS (EC2, Elastic Beanstalk)

Use Elastic Beanstalk CLI or manually set up EC2.

#Dockerizing Node.js Apps

Create Dockerfile:

FROM node:18-alpine
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
RUN npm run build
EXPOSE 3000
CMD ["node", "dist/index.js"]

Build and run:

docker build -t my-app .
docker run -p 3000:3000 my-app

#CI/CD with GitHub Actions

Create .github/workflows/ci.yml:

name: CI
on: [push]
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - uses: actions/setup-node@v2
        with:
          node-version: '18'
      - run: npm ci
      - run: npm test

#17. Advanced Topics

#Worker Threads

import { Worker } from 'worker_threads';

const worker = new Worker('./worker.js');
worker.postMessage('Hello');
worker.on('message', (msg) => console.log(msg));

#Child Processes

import { exec } from 'child_process';

exec('ls -la', (err, stdout, stderr) => {
  console.log(stdout);
});

#Native Addons

Write C++ addons using N-API.

#Streams Advanced

Pipelines and transform streams.

#GraphQL with Apollo Server

npm install @apollo/server graphql

import { ApolloServer } from '@apollo/server';
import { startStandaloneServer } from '@apollo/server/standalone';

const typeDefs = `#graphql
  type Book {
    title: String
    author: String
  }
  type Query {
    books: [Book]
  }
`;

const resolvers = {
  Query: {
    books: () => [{ title: 'The Hobbit', author: 'J.R.R. Tolkien' }],
  },
};

const server = new ApolloServer({ typeDefs, resolvers });
await startStandaloneServer(server, { listen: { port: 4000 } });

#Microservices Architecture

Use message brokers (RabbitMQ, Kafka) or HTTP APIs to communicate between services.

#18. Best Practices and Project Structure

#Folder Structure

src/
  ├── controllers/
  ├── models/
  ├── routes/
  ├── middleware/
  ├── services/
  ├── utils/
  ├── config/
  ├── types/
  └── app.ts
  └── server.ts
tests/
.env
package.json
tsconfig.json

#Coding Conventions

• Use ESLint and Prettier.
• Follow consistent naming (camelCase for variables, PascalCase for classes).
• Use async/await over callbacks.
• Document with JSDoc.

#Environment Variables Management

Use dotenv and validate with envalid or zod.

#Graceful Shutdown

process.on('SIGTERM', () => {
  console.log('SIGTERM received, shutting down gracefully');
  server.close(() => {
    console.log('HTTP server closed');
    // close database connections, etc.
    process.exit(0);
  });
});

#Monitoring and Alerting

Use tools like Prometheus, Grafana, New Relic, or Sentry.

#19. Real-World Project: Task Management API

#Project Overview

Build a RESTful API for task management with user authentication, CRUD operations, and database persistence.

#Setup and Dependencies

mkdir task-manager-api
cd task-manager-api
npm init -y
npm install express mongoose jsonwebtoken bcrypt dotenv zod
npm install -D typescript @types/node @types/express @types/mongoose @types/jsonwebtoken @types/bcrypt nodemon ts-node
npx tsc --init

#Project Structure

src/
  ├── models/
  │   ├── User.ts
  │   └── Task.ts
  ├── controllers/
  │   ├── authController.ts
  │   └── taskController.ts
  ├── routes/
  │   ├── authRoutes.ts
  │   └── taskRoutes.ts
  ├── middleware/
  │   ├── auth.ts
  │   └── errorHandler.ts
  ├── utils/
  │   └── validation.ts
  ├── config/
  │   └── database.ts
  ├── app.ts
  └── server.ts

#Database Models

// models/User.ts
import mongoose from 'mongoose';

const userSchema = new mongoose.Schema(
  {
    name: { type: String, required: true },
    email: { type: String, required: true, unique: true },
    password: { type: String, required: true },
  },
  { timestamps: true }
);

export const User = mongoose.model('User', userSchema);

// models/Task.ts
import mongoose from 'mongoose';

const taskSchema = new mongoose.Schema(
  {
    title: { type: String, required: true },
    description: { type: String },
    completed: { type: Boolean, default: false },
    user: { type: mongoose.Schema.Types.ObjectId, ref: 'User', required: true },
  },
  { timestamps: true }
);

export const Task = mongoose.model('Task', taskSchema);

#Authentication and Authorization

// middleware/auth.ts
import jwt from 'jsonwebtoken';
import { Request, Response, NextFunction } from 'express';

export const auth = async (req: Request, res: Response, next: NextFunction) => {
  try {
    const token = req.header('Authorization')?.replace('Bearer ', '');
    if (!token) throw new Error();
    const decoded = jwt.verify(token, process.env.JWT_SECRET!) as { userId: string };
    req.userId = decoded.userId;
    next();
  } catch (err) {
    res.status(401).json({ error: 'Please authenticate' });
  }
};

#API Endpoints

• POST /api/auth/register – Register user
• POST /api/auth/login – Login user
• GET /api/tasks – Get all tasks for authenticated user
• POST /api/tasks – Create task
• PUT /api/tasks/:id – Update task
• DELETE /api/tasks/:id – Delete task

#Validation and Error Handling

Use Zod for validation and centralized error handling middleware.

#Testing

Write tests with Jest and Supertest.

#Deployment

Deploy to Heroku, AWS, or using Docker.

#20. Real-World Project: Real-Time Chat Application

#Project Overview

Build a real-time chat application using Socket.io, with user authentication and message persistence.

#Setup and Dependencies

mkdir chat-app
cd chat-app
npm init -y
npm install express socket.io mongoose jsonwebtoken bcrypt dotenv
npm install -D typescript @types/node @types/express @types/socket.io @types/mongoose @types/jsonwebtoken @types/bcrypt nodemon ts-node
npx tsc --init

#Project Structure

src/
  ├── models/
  │   ├── User.ts
  │   └── Message.ts
  ├── controllers/
  │   └── authController.ts
  ├── middleware/
  │   └── auth.ts
  ├── sockets/
  │   └── chatSocket.ts
  ├── config/
  │   └── database.ts
  ├── app.ts
  └── server.ts

#Socket.io Integration

// server.ts
import http from 'http';
import { Server } from 'socket.io';
import app from './app';
import { setupChatSocket } from './sockets/chatSocket';

const server = http.createServer(app);
const io = new Server(server, {
  cors: { origin: 'http://localhost:3000' },
});

setupChatSocket(io);

server.listen(4000, () => {
  console.log('Server running on port 4000');
});

// sockets/chatSocket.ts
import { Server, Socket } from 'socket.io';
import { Message } from '../models/Message';

export const setupChatSocket = (io: Server) => {
  io.use(async (socket, next) => {
    try {
      const token = socket.handshake.auth.token;
      // verify token and attach user
      next();
    } catch (err) {
      next(new Error('Authentication error'));
    }
  });

  io.on('connection', (socket: Socket) => {
    console.log('User connected:', socket.id);

    socket.on('join room', (roomId: string) => {
      socket.join(roomId);
    });

    socket.on('chat message', async (data: { roomId: string; message: string }) => {
      // save message to database
      const message = new Message({
        roomId: data.roomId,
        userId: socket.data.userId,
        content: data.message,
      });
      await message.save();

      io.to(data.roomId).emit('chat message', {
        userId: socket.data.userId,
        content: data.message,
        timestamp: new Date(),
      });
    });

    socket.on('disconnect', () => {
      console.log('User disconnected:', socket.id);
    });
  });
};

#User Authentication

Similar to task manager, with JWT.

#Message Persistence

// models/Message.ts
import mongoose from 'mongoose';

const messageSchema = new mongoose.Schema(
  {
    roomId: { type: String, required: true },
    userId: { type: mongoose.Schema.Types.ObjectId, ref: 'User', required: true },
    content: { type: String, required: true },
  },
  { timestamps: true }
);

export const Message = mongoose.model('Message', messageSchema);

#Client-Side Integration

Provide a simple HTML/JS client or React app using Socket.io client.

#Deployment

Deploy to platforms supporting WebSockets (Heroku, AWS, DigitalOcean).

This guide covers the essential aspects of Node.js from beginner to advanced, with TypeScript examples throughout. Practice by building the real-world projects and refer to the official documentation for deeper dives. Happy coding!