This content originally appeared on DEV Community and was authored by Matt Angelosanto
Written by Subha Chanda ✏️
URL shorteners like Bitly and Cuttly are incredibly popular. In this article, we are going to create a similar tool by building an API service that shortens the URLs provided to it.
For the project, we are going to use MongoDB and Node.js, so you should have basic knowledge of them for this tutorial.
Planning the URL shortener build process in Node.js
Let's first plan out the building process, which is pretty straightforward. For each URL passed into our API, we will generate a unique ID and create a short URL with it. Then, the long URL, short URL, and unique ID will be stored in the database.
When a user sends a GET request to the short URL, the URL will be searched within the database, and the user will be redirected to the corresponding original URL. Sound complex? Don't worry, we'll cover everything you need to know.
Initializing the app and installing dependencies with MongoDB
First, we are going to need a database. Because we'll use MongoDB, we'll need a MongoDB SRV URI. You can create a database from this link. Our next step is to initialize the project folder with NPM.
Let's use the command npm init in the project directory to initialize. Once the project is initialized, we are going to install the required dependencies. The dependencies that we need are:
- dotenv: this package loads the environment variables from a file called
.envtoprocess.env - Express.js: this is a minimal and flexible web application framework for Node.js
- Mongoose: this is a MongoDB object modeling tool for Node.js
- ShortId: this package enables us to generate the short IDs for our URLs
The only developer dependency that we need is nodemon. nodemon is a simple tool that automatically restarts the Node.js server when a file change occurs.
Now, let's install the dependencies. To install the dependencies that we are going to need in our app, we will use the command:
npm i dotenv express mongoose shortid
After the dependencies are installed, we'll install the developer dependency:
npm i -D nodemon
Let's create our server in our app.js file using Express. To set up an Express server, we need to import the Express package into the app.js file. Once the package is imported, initialize and store it into a variable called app.
Now, use the available listen function to create the server. Here's an example.
const Express = require('Express');
const app = Express();
// Server Setup
const PORT = 3333;
app.listen(PORT, () => {
console.log(`Server is running at PORT ${PORT}`);
});
I've used port 3333 to run the server. The listen method in Express starts a UNIX socket and listens for a connection in a given port.
Now, create a .env file inside the config folder to store the MongoDB SRV URI and the base URL. The base URL will be your local host server location for now. Here's my .env file code:
MONGO_URI=mongodb+srv://nemo:YourPasswordHere@cluster0.mkws3.mongodb.net/myFirstDatabase?retryWrites=true&w=majority
BASE=http://localhost:3333
Remember to change the <password> field in the MongoDB URI with your database password.
Connecting the database to the app
Now, we'll connect the database to the app. To do so, import the Mongoose and dotenv dependencies into your db.js file, which is inside the config folder.
const mongoose = require('mongoose');
require('dotenv').config({ path: './.env' });
The path object key is passed inside the dotenv config because the .env file is not located in the root directory. We are passing the location of the .env file through this.
Now create an asynchronous function called connectDB within a file called db.js, inside the config folder. I'll use async/await for this article.
const connectDB = async () => {
try {
await mongoose.connect(process.env.MONGO_URI, {
useNewUrlParser: true,
useUnifiedTopology: true,
});
console.log('Database Connected');
} catch (err) {
console.error(err.message);
process.exit(1);
}
};
module.exports = connectDB;
In the try block, we wait for Mongoose to connect with the given MongoDB URI. The first parameter in the mongoose.connect method is the MongoDB SRV URI. Notice that the two key-value pairs are passed in the second parameter to remove the console warnings. Let's understand what the two key-value parameters mean.
-
useNewUrlParser: true: the underlying MongoDB driver has deprecated the current connection string parser. This is why it has added a new flag. If the connection encounters any issue with the new string parser, it can fall back to the old one -
useUnifiedTopology: true: this is set tofalseby default. Here, it is set totrueso that the MongoDB driver's new connection management engine can be used
If any error occurs within the catch statement, we will console log the error and exit with process.exit(1). Finally, we export the function with module.exports.
Now, import the db.js file into the app.js file with const connectDB = require('./config/db'); and call the connectDB function with connectDB().
Creating the Mongoose schema in MongoDB
We'll use a Mongoose schema to determine how data is stored in MongoDB. Essentially, the Mongoose schema is a model for the data. Let's create a file called Url.js inside a models folder. Import Mongoose here, then use the mongoose.Schema constructor to create the schema.
const mongoose = require('mongoose');
const UrlSchema = new mongoose.Schema({
urlId: {
type: String,
required: true,
},
origUrl: {
type: String,
required: true,
},
shortUrl: {
type: String,
required: true,
},
clicks: {
type: Number,
required: true,
default: 0,
},
date: {
type: String,
default: Date.now,
},
});
module.exports = mongoose.model('Url', UrlSchema);
The parent object keys are the keys that are going to be stored inside the database. We define each data key. Note that there is a required field for some and a default value for other keys.
Finally, we export the schema using module.exports = mongoose.model('Url', UrlSchema);. The first parameter inside mongoose.model is the singular form of the data that is to be stored, and the second parameter is the schema itself.
Building the URL and index routes
The URL route will create a short URL from the original URL and store it inside the database. Create a folder called routes in the root directory and a file named urls.js inside of it. We are going to use the Express router here. First, import all of the necessary packages, like so.
const Express = require('express');
const router = Express.Router();
const shortid = require('shortid');
const Url = require('../models/Url');
const utils = require('../utils/utils');
require('dotenv').config({ path: '../config/.env' });
The utils.js file inside the utils folder consists of a function that checks if a passed URL is valid or not. Here's the code for the utils.js file.
function validateUrl(value) {
return /^(?:(?:(?:https?|ftp):)?\\/\\/)(?:\\S+(?::\\S*)?@)?(?:(?!(?:10|127)(?:\\.\\d{1,3}){3})(?!(?:169\\.254|192\\.168)(?:\\.\\d{1,3}){2})(?!172\\.(?:1[6-9]|2\\d|3[0-1])(?:\\.\\d{1,3}){2})(?:[1-9]\\d?|1\\d\\d|2[01]\\d|22[0-3])(?:\\.(?:1?\\d{1,2}|2[0-4]\\d|25[0-5])){2}(?:\\.(?:[1-9]\\d?|1\\d\\d|2[0-4]\\d|25[0-4]))|(?:(?:[a-z\\u00a1-\\uffff0-9]-*)*[a-z\\u00a1-\\uffff0-9]+)(?:\\.(?:[a-z\\u00a1-\\uffff0-9]-*)*[a-z\\u00a1-\\uffff0-9]+)*(?:\\.(?:[a-z\\u00a1-\\uffff]{2,})))(?::\\d{2,5})?(?:[/?#]\\S*)?$/i.test(
value
);
}
module.exports = { validateUrl };
We will use the HTTP post request in the urls.js file to generate and post the details to the database.
const Express = require('express');
const router = Express.Router();
const shortid = require('shortid');
const Url = require('../models/Url');
const utils = require('../utils/utils');
require('dotenv').config({ path: '../config/.env' });
// Short URL Generator
router.post('/short', async (req, res) => {
const { origUrl } = req.body;
const base = process.env.BASE;
const urlId = shortid.generate();
if (utils.validateUrl(origUrl)) {
try {
let url = await Url.findOne({ origUrl });
if (url) {
res.json(url);
} else {
const shortUrl = `${base}/${urlId}`;
url = new Url({
origUrl,
shortUrl,
urlId,
date: new Date(),
});
await url.save();
res.json(url);
}
} catch (err) {
console.log(err);
res.status(500).json('Server Error');
}
} else {
res.status(400).json('Invalid Original Url');
}
});
module.exports = router;
The const { origUrl } = req.body; will extract the origUrl value from the HTTP request body. Then we store the base URL into a variable. const urlId = shortid.generate(); is generating and storing a short ID to a variable.
Once it is generated, we check if the original URL is valid using our function from the utils directory. For valid URLs, we move into the try block.
Here, we first search if the original URL already exists in our database with the Url.findOne({ origUrl }); Mongoose method. If found, we return the data in JSON format; otherwise, we create a short URL combining the base URL and the short ID.
Then, using our Mongoose model, we pass in the fields to the model constructor and save it to the database with the url.save(); method. Once saved, we return the response in JSON format.
Unexpected errors for the try block are handled in the catch block, and invalid URLs that return false in our validateUrl function send back a message that the URL is invalid. Finally, we export the router.
Previously, we needed to install the body-parser package, but now it is integrated into Express. So head back to the app.js file and add these two lines to use body-parser:
// Body Parser
app.use(Express.urlencoded({ extended: true }));
app.use(Express.json());
These two lines help us read incoming requests. After these two lines of code, import the URL route.
app.use('/api', require('./routes/urls'));
Because we are using the /api endpoint, our complete endpoint becomes http://localhost:3333/api/short. Here's an example.
Now create another file called index.js inside the routes folder to handle the redirection process. In this file, import the necessary dependencies.
Here, we are first going to search our database for the short URL ID that is passed. If the URL is found, we'll redirect to the original URL.
const Express = require('express');
const router = Express.Router();
const Url = require('../models/Url');
router.get('/:urlId', async (req, res) => {
try {
const url = await Url.findOne({ urlId: req.params.urlId });
if (url) {
url.clicks++;
url.save();
return res.redirect(url.origUrl);
} else res.status(404).json('Not found');
} catch (err) {
console.log(err);
res.status(500).json('Server Error');
}
});
module.exports = router;
The HTTP GET request is getting the URL ID with the help of :urlId. Then, inside the try block, we find the URL using the Url.findOne method, similar to what we did in the urls.js route.
If the URL is found, we increase the number of clicks to the URL and save the click amount. Finally, we redirect the user to the original URL using return res.redirect(url.origUrl);.
If the URL is not found, we send a JSON message that the URL is not found. Any uncaught exception is handled in the catch block. We console log the error and send a JSON message of "Server Error". Finally, we export the router.
Import the route to the app.js file, and our URL shortener is ready to use. After importing it, our final app.js file will look like this:
const Express = require('Express');
const app = Express();
const connectDB = require('./config/db');
require('dotenv').config({ path: './config/.env' });
connectDB();
// Body Parser
app.use(Express.urlencoded({ extended: true }));
app.use(Express.json());
app.use('/', require('./routes/index'));
app.use('/api', require('./routes/urls'));
// Server Setup
const PORT = 3333;
app.listen(PORT, () => {
console.log(`Server is running at PORT ${PORT}`);
});
Conclusion
In this article, we learned how to build a URL shortening service API from scratch. You can integrate it with any frontend you want, and even build a full-stack URL shortener service. I hope you liked reading this article and learned something new along the way. You can find the complete source code on my GitHub repo.
200’s only ✔️ Monitor failed and slow network requests in production
Deploying a Node-based web app or website is the easy part. Making sure your Node instance continues to serve resources to your app is where things get tougher. If you’re interested in ensuring requests to the backend or third party services are successful, try LogRocket.
LogRocket is like a DVR for web apps, recording literally everything that happens on your site. Instead of guessing why problems happen, you can aggregate and report on problematic network requests to quickly understand the root cause.
LogRocket instruments your app to record baseline performance timings such as page load time, time to first byte, slow network requests, and also logs Redux, NgRx, and Vuex actions/state. Start monitoring for free.
This content originally appeared on DEV Community and was authored by Matt Angelosanto
Matt Angelosanto | Sciencx (2021-07-06T20:42:53+00:00) Building a URL shortener with Node.js. Retrieved from https://www.scien.cx/2021/07/06/building-a-url-shortener-with-node-js/
Please log in to upload a file.
There are no updates yet.
Click the Upload button above to add an update.