DEV Community

Joan Roucoux
Joan Roucoux

Posted on

Building a Scalable URL Shortener with Node.js (Part 1/2)

#node #redis #mongodb #zookeeper

Introduction

I'm sure you're familiar with URL shortener tools like TinyURL and Bitly, as they are widely used online. It simply takes a long URL and creates a shorter, unique alias that redirects to the original link.

For example, a URL like https://www.example.com/movies/avengers-infinity-war/casting/chris-hemsworth could be shortened to something like https://short.url/abc123, making it much easier to read and share. And then, when the user accesses the shortened link, the browser requests the server, which looks up the alias in its database and redirects the user to the original URL.

This might look simple when you first think about it, but it's actually tricky to build which is why they are commonly used in system design interviews. So creating one yourself is a great way to practice and see the challenges involved (handling high traffic, concurrency and race conditions in key generation, data storage and scaling...).

In the first part of this tutorial, we will focus on developing the backend of our URL shortener application using the following stack:

  • Node.js: A JavaScript runtime to power the server instances.
  • MongoDB: A NoSQL database for storing original URLs.
  • Redis: In-memory data store for caching frequently accessed URLs.
  • Apache ZooKeeper: A centralized service to generate unique IDs and prevent race conditions between instances.
  • Nginx: A load balancer and reverse proxy to distribute traffic across server instances.
  • Docker: A containerization tool to manage all the services.

Here's the high-level architecture of the application:

URL Shortener App High Level Architecture

So let's jump into it 🚀

Initializing the Project

First, let's see what our file tree will look like:

url-shortener-demo-app ├─ .env ├─ docker-compose.yml ├─ client - Part 2 of this tutorial ├─ nginx │ ├─ Dockerfile │ └─ nginx.conf └─ server ├─ src │ ├─ config │ │ ├─ mongoose.ts │ │ ├─ redis.ts │ │ └─ zookeeper.ts │ ├─ controllers │ │ └─ urlsController.ts │ ├─ models │ │ └─ Url.ts │ ├─ repositories │ │ └─ urlsRepository.ts │ ├─ routes │ │ └─ urlsRoutes.ts │ ├─ services │ │ └─ urlsService.ts │ ├─ utils │ │ └─ index.ts │ └─ index.ts ├─ .dockerignore ├─ Dockerfile ├─ package-lock.json ├─ package.json └─ tsconfig.json
Enter fullscreen mode Exit fullscreen mode

Our project app url-shortener-demo-app will include a client folder for the React app that we will build in the second part of this tutorial, a nginx folder for Nginx configuration and a server folder for managing URL-related operations. It's a good thing to separate concepts like routers, controllers and services to enhance code organization and improve maintainability.

We will first start with the server code. Create a new directory for the project and initialize a new application:

mkdir url-shortener-demo-app cd url-shortener-demo-app mkdir server cd server npm init -y
Enter fullscreen mode Exit fullscreen mode

Then, install all the required dependencies:

npm install fastify @fastify/cors mongoose ioredis zookeeper npm install --save-dev typescript @types/node ts-node
Enter fullscreen mode Exit fullscreen mode

The packages you installed above include:

  • fastify: A simple web framework for handling routes and middleware.
  • @fastify/cors: Middleware for enabling Cross-Origin Resource Sharing (CORS).
  • zookeeper: A client for interacting with Apache ZooKeeper.
  • mongoose: An Object Data Modeling (ODM) library for MongoDB.
  • ioredis: A client for interacting with Redis.
  • And some devDependencies for Typescript support.

Next, open the package.json file and a in the scripts section a new script to run the application:

... "scripts": { "start": "ts-node src/index.ts" }, ...
Enter fullscreen mode Exit fullscreen mode

And finally, create a tsconfig.json file for Typescript configuration by running:

tsc --init
Enter fullscreen mode Exit fullscreen mode

Great, we can now move on and add the configuration files for MongoDB, Redis and ZooKeeper.

Setting Up MongoDB

Using mongoose in our application makes it easier to create and manage data within MongoDB, providing a more convenient and structured approach.

In src/config/mongoose.ts, add the following code to configure our MongoDB connection:

import { connect } from 'mongoose'; const { MONGODB_USER, MONGODB_PASSWORD, MONGODB_DATABASE, MONGODB_HOST, MONGODB_DOCKER_PORT, } = process.env; const MONGO_URI = `mongodb://${MONGODB_USER}:${MONGODB_PASSWORD}@${MONGODB_HOST}:${MONGODB_DOCKER_PORT}/${MONGODB_DATABASE}?authSource=admin`; // Connect to MongoDB export const connectToMongoDB = async (): Promise<void> => { try { await connect(MONGO_URI); console.log('Successfully connected to MongoDB'); } catch (error) { console.error('Error connecting to MongoDB:', error); throw error; } };
Enter fullscreen mode Exit fullscreen mode

We're loading some environment variables from the .env file in the root directory, so be sure to create it and add the following keys:

MONGODB_USER=user MONGODB_PASSWORD=pass MONGODB_DATABASE=urls MONGODB_HOST=mongo MONGODB_LOCAL_PORT=27017 MONGODB_DOCKER_PORT=27017
Enter fullscreen mode Exit fullscreen mode

Then, create the URL data model, which will define the structure of our documents in the database. This model will include the following attributes:

  • originalUrl: The original URL that users want to shorten.
  • shortenUrlKey: A unique identifier generated for the shortened URL.
  • createdAt: A timestamp indicating when the URL was created.
  • expiresAt: A timestamp indicating when the shortened URL will expire.

In src/models/Url.ts, add the following code:

import { Document, Schema, model } from 'mongoose'; export interface IUrl extends Document { originalUrl: string; shortenUrlKey: string; createdAt: Date; expiresAt: Date; } const schema = new Schema<IUrl>({ originalUrl: { type: String, required: true, unique: true, }, shortenUrlKey: { type: String, required: true, unique: true, }, createdAt: { type: Date, default: new Date(), }, expiresAt: { type: Date, default: new Date(new Date().setMinutes(new Date().getMinutes() + 10)), // default is 10 minutes, for demonstration only }, }); export default model<IUrl>('url', schema);
Enter fullscreen mode Exit fullscreen mode

Setting Up Redis

In the same way that we use mongoose to interact with our MongoDB database, we will use ioredis client to interact with Redis and manage frequently accessed URLs by leveraging caching mechanisms for faster retrieval.

We will override some of the initial Redis methods to:

  • Add a new entry in Redis cache with set().
  • Retrieve a Redis cache entry with get().
  • Extend TTL of an existing entry with extendTTL().

In src/config/redis.ts, add the following code:

import { Redis } from 'ioredis'; const { REDIS_HOST, REDIS_DOCKER_PORT } = process.env; export enum RedisExpirationMode { EX = 'EX', // Expire in seconds } let client: Redis | null; // Get Redis client const getRedisClient = (): Redis => { if (!client) { const config = { host: REDIS_HOST, port: Number(REDIS_DOCKER_PORT), maxRetriesPerRequest: null, }; client = new Redis(config); } return client; }; // Connect to Redis export const connectToRedis = async (): Promise<void> => { const client = getRedisClient(); client .on('connect', () => { console.log('Successfully connected to Redis'); }) .on('error', (error) => { console.error('Error on Redis:', error.message); }); }; // Set a key/value pair export const set = async ( key: string, value: string, expirationMode: RedisExpirationMode, seconds: number ): Promise<void> => { try { await getRedisClient().set(key, value, expirationMode, seconds); console.info(`Key ${key} created in Redis cache`); } catch (error) { console.error(`Failed to create key in Redis cache: ${error}`); } }; // Get a value from a key export const get = async (key: string): Promise<string | null> => { try { const value = await getRedisClient().get(key); console.info(`Value with key ${key} retrieved from Redis cache`); return value; } catch (error) { console.error( `Failed to retrieve value with key ${key} in Redis cache: ${error}` ); return null; } }; // Extend TTL of a key export const extendTTL = async ( key: string, additionalTimeInSeconds: number ) => { // Get the current TTL of the key const currentTTL = await getRedisClient().ttl(key); if (currentTTL > 0) { // Calculate the new TTL const newTTL = currentTTL + additionalTimeInSeconds; // Set the new TTL await getRedisClient().expire(key, newTTL); console.info(`TTL for key ${key} extended to ${newTTL} in Redis cache`); } else { console.error(`Failed to extend TTL of key ${key} in Redis cache`); } };
Enter fullscreen mode Exit fullscreen mode

Finally, add the following keys to the .env file to configure Redis service:

REDIS_HOST=redis REDIS_LOCAL_PORT=6379 REDIS_DOCKER_PORT=6379
Enter fullscreen mode Exit fullscreen mode

Setting Up ZooKeeper

Apache ZooKeeper will help us avoid race conditions by making sure that only one node generates a token at a time, maintaining data integrity.

One approach could be to assign each server registered to the ZooKeeper service a specific token range and generate a token within that range. However, I chose a simpler solution: checking if a token already exists under the /tokens path. If the token is not found, it will attempt to generate a new token repeatedly until it finds one that is available.

For instance, if /tokens/existingToken already exists, it will try again and register /tokens/newToken if available. In our example, we will use a token that is 6 characters long, which gives us around 69 billion possibilities (64^6). This should provide a comfortable buffer before we encounter any collisions in our demo app.

First, add a method to generate a base64 token in src/utils/index.ts:

import { randomBytes } from 'crypto'; export const generateBase64Token = (length: number): string => { const buffer = randomBytes(Math.ceil((length * 3) / 4)); // Generate enough random bytes return buffer .toString('base64') // Convert to Base64 .replace(/\+/g, '-') // URL-safe: replace + with - .replace(/\//g, '_') // URL-safe: replace / with _ .replace(/=+$/, '') // Remove padding .slice(0, length); // Ensure fixed length };
Enter fullscreen mode Exit fullscreen mode

Next, in src/config/zookeeper.ts, add the following code to:

  • Connect the client to ZooKeeper.
  • Create the /tokens node if it doesn't exist.
  • Generate the unique token using the generateBase64Token() method we just created. 
import ZooKeeper from 'zookeeper'; import { generateBase64Token } from '../utils'; const { ZOOKEEPER_HOST, ZOOKEEPER_DOCKER_PORT } = process.env; const host = `${ZOOKEEPER_HOST}:${ZOOKEEPER_DOCKER_PORT}`; let client: ZooKeeper | null; const TOKENS_NODE_PATH = '/tokens'; const MAX_RETRIES = 3; const MAX_TOKEN_SIZE = 6; // Get ZooKeeper client const getZookeeperClient = (): ZooKeeper => { if (!client) { const config = { connect: host, timeout: 5000, debug_level: ZooKeeper.constants.ZOO_LOG_LEVEL_WARN, host_order_deterministic: false, }; client = new ZooKeeper(config); } return client; }; // Connect to ZooKeeper export const connectToZookeeper = async (): Promise<void> => { const client = getZookeeperClient(); await new Promise<void>((resolve, reject) => { client.connect(client.config, async (error) => { if (error) { console.error('Error connecting to ZooKeeper:', error); reject(); } console.log('Successfully connected to ZooKeeper'); await createTokensNode(); resolve(); }); }); }; // Create '/tokens' node if it doesn't exist const createTokensNode = async (): Promise<void> => { const client = getZookeeperClient(); const doesTokensNodeExist = await client.pathExists(TOKENS_NODE_PATH, false); // If it does, do nothing if (doesTokensNodeExist) { console.info(`Tokens node ${TOKENS_NODE_PATH} already exists`); return; } // If it doesn't exist, create the root path await new Promise<void>((resolve, reject) => { client.mkdirp(TOKENS_NODE_PATH, (error) => { if (error) { console.error(`Failed to create tokens node: ${error}`); reject(); } console.info(`Tokens node ${TOKENS_NODE_PATH} created`); resolve(); }); }); }; // Create a node const createNode = async (path: string, data: Buffer): Promise<void> => { try { await getZookeeperClient().create( path, data, ZooKeeper.constants.ZOO_EPHEMERAL ); console.info(`Node ${path} created`); } catch (error) { console.error(`Failed to create node: ${error}`); throw error; } }; // Generate a unique token with retries for collision detection export const generateUniqueToken = async (retryCount = 0): Promise<string> => { const client = getZookeeperClient(); const token = generateBase64Token(MAX_TOKEN_SIZE); const uniqueTokenPath = `${TOKENS_NODE_PATH}/${token}`; // Create a child node with the generated token try { // Check if the unique token node already exists const doesUniqueTokenNodeExist = await client.pathExists( uniqueTokenPath, false ); // If it does, retry if (doesUniqueTokenNodeExist) { if (retryCount < MAX_RETRIES) { console.log( `Token collision detected for path: ${uniqueTokenPath}. Retrying... Attempt ${ retryCount + 1 } of ${MAX_RETRIES}` ); return await generateUniqueToken(retryCount + 1); } else { throw new Error( `Failed to generate a unique token after ${MAX_RETRIES} attempts due to collisions.` ); } } // If it doesn't exist, create the node await createNode(uniqueTokenPath, Buffer.from(token)); return token; // Return the unique token on success } catch (error) { console.error(`Error generating the unique token node: ${error}`); throw error; } };
Enter fullscreen mode Exit fullscreen mode

And finally, add the following keys to the .env file to configure the service:

ZOOKEEPER_HOST=zookeeper ZOOKEEPER_LOCAL_PORT=2181 ZOOKEEPER_DOCKER_PORT=2181
Enter fullscreen mode Exit fullscreen mode

Great, we have all of our configuration files now ready! We can move on and start implementing the actual URL logic.

Implementing the URL Shortening Logic

In this section, we will walk through the process of implementing the logic needed to shorten a URL and manage the redirection when users access the shortened link.

Implementing URL repository

The URL repository will handle all database operations for managing shortened URLs:

  • Add a new URL to the database with create().
  • Retrieve all saved URLs with findAll().
  • Fetch a specific URL based on given parameters with findOne().

Add the following code in src/repositories/urlRepository.ts:

import Url, { IUrl } from '../models/Url'; interface ICreateParams { shortenUrlKey: string; originalUrl: string; } interface IFindOneParams { shortenUrlKey?: string; originalUrl?: string; } // Create a shortened URL const create = async (params: ICreateParams): Promise<IUrl> => { console.log(`Creating URL with params: ${JSON.stringify(params)}`); const result: IUrl = await Url.create({ ...params }); console.log(`Created URL: ${JSON.stringify(result)}`); return result; }; // Find all URLs const findAll = async (): Promise<IUrl[]> => { console.log('Finding all URLs'); const result: IUrl[] = await Url.find(); console.log(`Found URLs: ${result?.length || 0}`); return result; }; // Find a specific URL const findOne = async (params: IFindOneParams): Promise<IUrl | null> => { console.log(`Finding one URL with params: ${JSON.stringify(params)}`); const result: IUrl | null = await Url.findOne({ ...params }); console.log(`Found URL: ${JSON.stringify(result)}`); return result; }; export { create, findAll, findOne };
Enter fullscreen mode Exit fullscreen mode

Implementing URL validation

Before shortening a URL, it's essential to verify that the input provided by the user is valid. We will use a simple Regex found online for it (you can find plenty of other patterns as well depending on your needs).

Add the following method to src/utils/index.ts:

... export const isValidUrl = (value: string): boolean => { const pattern: RegExp = new RegExp( '^https?:\\/\\/' + // Protocol (http or https) '(?:www\\.)?' + // Optional www. '[-a-zA-Z0-9@:%._\\+~#=]{1,256}' + // Domain name characters '\\.[a-zA-Z0-9()]{1,6}\\b' + // Top-level domain '(?:[-a-zA-Z0-9()@:%_\\+.~#?&\\/=]*)$', // Optional query string 'i' // Case-insensitive flag ); return pattern.test(value); };
Enter fullscreen mode Exit fullscreen mode

Implementing Service Logic

The service layer is responsible for orchestrating the business logic of the URL shortener application, acting as an intermediary between the controller and the repository we just created.

As mentioned in the Redis section, we will leverage Redis for caching frequently accessed URLs to improve performance by reducing database queries.

In src/services/urlService.ts, add the following code:

import { generateUniqueToken } from '../config/zookeeper'; import { get, set, extendTTL, RedisExpirationMode } from '../config/redis'; import { IUrl } from '../models/Url'; import { isValidUrl } from '../utils'; import { create, findAll, findOne } from '../repositories/urlsRepository'; const ONE_MINUTE_IN_SECONDS = 60; // Get all shortened URLs export const getAllUrls = async (): Promise<IUrl[]> => await findAll(); // Get a specific shortened URL by its key export const getUrlByShortenUrlKey = async ( shortenUrlKey: string ): Promise<string | null> => { // Try to get the original URL from Redis cache const cachedOriginalUrl = await get(shortenUrlKey); if (cachedOriginalUrl) { // Extend TTL await extendTTL(shortenUrlKey, ONE_MINUTE_IN_SECONDS); return cachedOriginalUrl; // Return the cached original URL } // If not in cache, retrieve from database const savedUrl = await findOne({ shortenUrlKey }); if (savedUrl) { // Cache the original URL created by its shorten URL key await set( savedUrl.shortenUrlKey, savedUrl.originalUrl, RedisExpirationMode.EX, ONE_MINUTE_IN_SECONDS ); return savedUrl.originalUrl; // Return the saved original URL } return null; // Return null if nothing found }; // Create a new shortened URL export const createShortenedUrl = async ( originalUrl: string ): Promise<string | null> => { // Check if URL is valid if (!isValidUrl(originalUrl)) { return null; } // Retrieve from database const savedUrl = await findOne({ originalUrl }); if (savedUrl) { return savedUrl.shortenUrlKey; // Return the saved shortened URL key } // If not in database, generate a new shortened URL key and save it const shortenUrlKey = await generateUniqueToken(); if (shortenUrlKey) { const newUrl = await create({ originalUrl, shortenUrlKey, }); // Cache the original URL created by its shorten URL key await set( newUrl.shortenUrlKey, newUrl.originalUrl, RedisExpirationMode.EX, ONE_MINUTE_IN_SECONDS ); return newUrl.shortenUrlKey; // Return shortened URL key } return null; // Return null if token generation failed };
Enter fullscreen mode Exit fullscreen mode

Implementing Controller Logic

Next, let's implement the URL controller methods to manage the HTTP status codes and return appropriate messages for our operations. As you might have seen, I chose to return a 200 status code along with the original URL (and not a 301 redirect) in the getUrl() method to prevent any CORS issues between the client and the requested URLs later on.

In src/controllers/urlController.ts, add the following code:

import { FastifyReply, FastifyRequest } from 'fastify'; import { createShortenedUrl, getAllUrls, getUrlByShortenUrlKey, } from '../services/urlsService'; // Get all shortened URLs export const getUrls = async ( _request: FastifyRequest, reply: FastifyReply ): Promise<void> => { try { const urls = await getAllUrls(); return reply.code(200).send(urls); } catch (error) { return reply .code(500) .send('Failed to retrieve the list of URLs. Please try again later'); } }; // Get a specific URL by its key export const getUrl = async ( request: FastifyRequest<{ Params: { shortenUrlKey: string; }; }>, reply: FastifyReply ): Promise<void> => { try { const { shortenUrlKey } = request.params; const originalUrl = await getUrlByShortenUrlKey(shortenUrlKey); if (!originalUrl) { return reply .code(404) .send('The requested shortened URL could not be found'); } return reply.code(200).send(originalUrl); } catch (error) { return reply.code(500).send('Unable to retrieve the specified URL'); } }; // Create a new shortened URL export const postUrl = async ( request: FastifyRequest<{ Body: { originalUrl: string; }; }>, reply: FastifyReply ): Promise<void> => { try { const { originalUrl } = request.body; const shortenUrlKey = await createShortenedUrl(originalUrl); if (!shortenUrlKey) { return reply.code(400).send('The provided URL is invalid'); } return reply.code(201).send(shortenUrlKey); } catch (error) { return reply.code(500).send('Failed to create a shortened URL'); } };
Enter fullscreen mode Exit fullscreen mode

Implementing Routing Logic

Now, let's register the routes under the /urls prefix.

In src/routes/urls.ts, add the following code:

import { FastifyInstance } from 'fastify'; import { postUrl, getUrls, getUrl } from '../controllers/urlsController'; export const urlsRoutes = async (fastify: FastifyInstance) => { fastify.register( async (router: FastifyInstance) => { // Get all shortened URLs router.get('/', getUrls); // Get a specific URL by its key router.get('/:shortenUrlKey', getUrl); // Create a new shortened URL router.post('/', postUrl); }, { prefix: '/urls' } ); };
Enter fullscreen mode Exit fullscreen mode

Setting Up The Server

And finally, let's create a src/index.ts to set up the Fastify server and:

  • Configure CORS.
  • Register the URL router under the /api prefix.
  • Connect MongoDB, Redis, and ZooKeeper.
  • Start the server. 
import Fastify, { FastifyInstance } from 'fastify'; import fastifyCors from '@fastify/cors'; import { connectToMongoDB } from './config/mongoose'; import { connectToRedis } from './config/redis'; import { connectToZookeeper } from './config/zookeeper'; import { urlsRoutes } from './routes/urlsRoutes'; // Fastify server instance const fastify = Fastify(); // Configure server fastify .register(fastifyCors) // Register CORS .register( async (fastify: FastifyInstance) => { fastify.register(urlsRoutes); // Register URL routes }, { prefix: '/api' } ); // Start the server const start = async () => { try { // Connect to MongoDB, Redis and ZooKeeper await connectToMongoDB(); await connectToRedis(); await connectToZookeeper(); // Start Fastify server await fastify.listen({ port: Number(process.env.NODE_SERVER_LOCAL_PORT), host: process.env.NODE_SERVER_HOST, }); console.log('Server is now listening'); } catch (error) { console.error('Failed to start server:', error); process.exit(1); } }; start();
Enter fullscreen mode Exit fullscreen mode

Don't forget to add the following keys to the .env file:

NODE_SERVER_HOST=0.0.0.0 NODE_SERVER_LOCAL_PORT=3000
Enter fullscreen mode Exit fullscreen mode

Awesome, our server is now ready! We can move on and configure Nginx.

Setting Up Nginx

As mentioned in the introduction, we will use Nginx as a load balancer and reverse proxy to distribute traffic across server instances and improve response times. We will use the default round-robin algorithm, which is ideal for distributing requests evenly and making our application more resilient.

In a nginx/nginx.conf file, add the following configuration:

  • Create an upstream node_servers block to define the group of servers listening on local port 3000. You will see later in the docker-compose setup that we did not define a specific port for each server instance, allowing Docker to dynamically assign ports and manage load balancing.
  • Add a server block which listens on port 80 for incoming requests.
  • Include a location block to forward requests made on the /api/ path to the node_servers group using proxy_pass. And add proxy_set_header directives to ensure that client request details are forwarded to the servers. 
upstream node_servers { server server:$NODE_SERVER_LOCAL_PORT; } server { listen $NGINX_DOCKER_PORT; # Serve backend location /api/ { proxy_pass http://node_servers/api/; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header X-Forwarded-Port $server_port; } }
Enter fullscreen mode Exit fullscreen mode

Also add the following keys to the .env file to configure the Nginx service:

NGINX_HOST=localhost NGINX_LOCAL_PORT=80 NGINX_DOCKER_PORT=80
Enter fullscreen mode Exit fullscreen mode

Containerization with Docker

With everything set up, we will now use Docker to containerize all of our services.

Let's first dockerize the server application. Create a Dockerfile in the /server folder that sets up the Node environment, installs dependencies, and starts the app like below:

# Use an official Node runtime as the base image FROM node:22.11.0 # Set the working directory WORKDIR /usr/src/server # Copy package.json and package-lock.json to the container COPY package*.json ./ # Install application dependencies RUN npm install # Copy the rest of the application code COPY . . # Run the application CMD [ "npm", "run", "start" ]
Enter fullscreen mode Exit fullscreen mode

Also add a .dockerignore file with node_modules inside because some libraries like zookeeper can cause some issues when compiled on different OS.

Next, create a Dockerfile in the nginx folder to configure Nginx by using the official Nginx image, copying our custom nginx.conf file to the container, and running the service:

# Use an official Nginx as the base image FROM nginx:stable-alpine # Copy nginx.conf to the container COPY nginx.conf /etc/nginx/templates/default.conf.template # Run the server CMD ["nginx", "-g", "daemon off;"]
Enter fullscreen mode Exit fullscreen mode

Finally, create a docker-compose.yml file at the root of your project setting up the services (MongoDB, Redis and ZooKeeper), while building the server and Nginx from their respective Dockerfiles:

services: mongo: image: mongo:latest environment: - MONGO_INITDB_ROOT_USERNAME=${MONGODB_USER} - MONGO_INITDB_ROOT_PASSWORD=${MONGODB_PASSWORD} ports: - ${MONGODB_LOCAL_PORT}:${MONGODB_DOCKER_PORT} volumes: - ./mongodb:/data/db redis: image: redis:latest ports: - ${REDIS_LOCAL_PORT}:${REDIS_DOCKER_PORT} zookeeper: image: zookeeper:latest ports: - ${ZOOKEEPER_LOCAL_PORT}:${ZOOKEEPER_DOCKER_PORT} server: depends_on: - mongo - redis - zookeeper environment: - MONGODB_USER=${MONGODB_USER} - MONGODB_PASSWORD=${MONGODB_PASSWORD} - MONGODB_DATABASE=${MONGODB_DATABASE} - MONGODB_HOST=${MONGODB_HOST} - MONGODB_DOCKER_PORT=${MONGODB_DOCKER_PORT} - REDIS_HOST=${REDIS_HOST} - REDIS_DOCKER_PORT=${REDIS_DOCKER_PORT} - ZOOKEEPER_HOST=${ZOOKEEPER_HOST} - ZOOKEEPER_DOCKER_PORT=${ZOOKEEPER_DOCKER_PORT} - NODE_SERVER_HOST=${NODE_SERVER_HOST} - NODE_SERVER_LOCAL_PORT=${NODE_SERVER_LOCAL_PORT} build: context: ./server dockerfile: Dockerfile volumes: - ./server:/usr/src/server - /usr/src/server/node_modules deploy: mode: replicated replicas: 3 nginx: depends_on: - server environment: - NODE_SERVER_LOCAL_PORT=${NODE_SERVER_LOCAL_PORT} - NGINX_DOCKER_PORT=${NGINX_DOCKER_PORT} build: context: ./nginx dockerfile: Dockerfile ports: - ${NGINX_LOCAL_PORT}:${NGINX_DOCKER_PORT}
Enter fullscreen mode Exit fullscreen mode

As you can see, we are passing down all the environment variables defined in the .env file, keeping all configurations centralized in one place, which makes adjustments simple.

Testing and Deployment

Run docker compose up -d to start all containers in detached mode. Once they're running, you can use cURL or any other tool to test the API routes:

  • Save a new URL (you can also test an incorrect URL format to check if the service returns a 400 status code): 
curl --location 'http://localhost/api/urls' \ --header 'Content-Type: application/json' \ --data '{ "originalUrl": "URL_HERE" }'
Enter fullscreen mode Exit fullscreen mode
  • Get all URLs: 
curl --location 'http://localhost/api/urls'
Enter fullscreen mode Exit fullscreen mode
  • Retrieve the original URL: 
curl --location 'http://localhost/api/urls/SHORTENED_TOKEN_HERE'
Enter fullscreen mode Exit fullscreen mode

Or use Postman which is more friendly:

Get all URLs in Postman

You can also check the logs in Docker to monitor the activity across the containers (I'm using Docker Desktop here):

Logs Docker Desktop

Conclusion

You have reached the end of the first part of this tutorial! I hope you enjoyed 😄

We learned how to build a scalable URL shortener application from scratch using Node.js, Redis, MongoDB, Apache ZooKeeper, Nginx and Docker. You can find the complete code for this project here.

In the second part, we will focus on developing the frontend of our application using React and RTK Query, allowing users to interact with our servers through a minimal UI.

And if you're interested in going further, check out my other repository here. In this version, I've added extra features like a visit counter and a purge system to clean all expired URLs, all managed through a task queue service.

Top comments (0)