Write a REST API in Golang following best practices

This content originally appeared on DEV Community and was authored by Lucas Neves Pereira

Hello there! Lately I have noticed that a lot of enterprises are starting to migrating their code base previously in Java, Python, C++ to Golang, specially for those using microservices.

Therefore writing a REST API in Go is surely a skill that will be on demand, but what triggered me to write this article was to show how to build such API using the best practices.

To follow those practices I had mainly two sources:

• One of them was the GitHub repository for the Golang Standard for project layout.

• Another one was this video of Mat Ryer that explains how he writes Golang HTTP services. I did not followed every rule he had but he really inspired me to write this API.

Let's do this ??

In this article, as an example I'll be building a very simple Students REST API. I'll try not to add a bunch of fields and methods to keep it simple and stay as minimalist as possible, still following the best practices, the goal here is really to inspire you to write and structure your Golang services in the best way possible ?

Structure the project

Start by creating a folder in your machine, mkdir students-api, and change your directory to that folder, cd students-api.

Now let's init a new go module by running go mod init students-api , this should create a go.mod file and now we can open your directory in your favourite code editor.

Then if I want to follow the Golang project layout standards I should create a cmd/server directory where I'll have my main.go that is my entry point for my app.

You may be wondering why we have a subdirectory server/ if we are already in a cmd/ directory, this is well thought to handle scalability to our project.
For example, if later we want to build a cli for our API, in that case we could just add a cmd/cli directory without messing with our functions in cmd/server/main.go . If you're seeking more information about this, just check the golang standards GitHub repository.

In our main.go file let's declare our package main and write a func Run() that will run our application and return and error if there is one.

Our students API is now correctly initialised and we can move on by creating our first endpoint.

First Endpoint

The first endpoint will be a status endpoint that will check if our API is up or not. Creating and endpoint api/status means using http, let's once again follow the project layout best practices and create and internal/ directory that will contain all of the packages and code that is private, meaning the code that we don't really want other to import into their code base. Most of our code for this project will be in our internal directory.

Inside of internal/ let's create a package http that will contain a handler.go file where we will "handle" all of our http related logic.

⚠️ Attention! In the screenshot above I've made a mistake and called the package handler instead of http.

In this handler.go file we're going to define a struct Handler that will store pointers to our services later and we will had 2 more function.

• A NewHandler() func that will insatiate the Handler.
• A InitRoutes() func that will initialise our endpoints.

I'll also be using the gorilla/mux router in this project because it saves us a bit of code with the standard net/http package. So in our struct will be adding a Router that will be a pointer to the gorilla/mux router.

Make sure to run a go get -u github.com/gorilla/mux to add it as a dependency.

⚠️ Attention! In the screenshot above I've made a mistake and called the package handler instead of http.

The function InitRoutes() will be a receiver function of our Handler, if you are coming from an Object Oriented Programming background just think of Handleras a class and InitRoutes() as a method of the Handler class. Moving on, since it's a receiver function of Handler we have access to it so we can call Router and set it to mux.NewRouter().
Then we can use our Handler Router and call HandleFunc like we would with the standard net/http package but we are using the gorilla/mux router.
In the HandleFunc method we pass the path of our status endpoint and as a second argument an anonymous function or literal function that has a response writer and a pointer to our request as argument. Once we hit the endpoint, we are just printing to the response writer "Status Up".

⚠️ Attention! Rename the package from handler to http.

To test this endpoint we need to go back to our cmd/server/main.go file and import our internal http package (we can add an alias to "rename" the import). Once our package is imported we can call our NewHandler() method then init our routes and finally ListenAndServe using the net/http package to ListenAndServe in a port of our choice passing in our internal Handler Router as a second argument. If something goes wrong we just return an error and our main() func will handle it.

Run the server with go run cmd/server/main.go

And in a different terminal window test the endpoint by running:

curl http://localhost:9000/api/status

Our first endpoint works and our students api is up! ??

The Database

It is time to implement our database package. In this project I've choose to use PostgreSQL and I'll be using Docker to to set it up locally. I'm assuming you have Docker and PostgreSQL in your machine for this tutorial.

If you are not using Docker and you prefer to use PostgreSQL locally on MacOS, i recommend Postico that is a really nice client for postgre that I enjoy using.

Head over to a terminal window and run the following command:

docker run ----name students-db ----env POSTGRES_PASSWORD=postgres -p 5432:5432 -d postgres

This will basically fetch a PostgreSQL image on docker hub and run it on your machine. You can run a docker ps to make sure your container is running on detached mode. By the way, if you want to stop it you just have to run docker stop students-db and docker rm students-db to actually remove it.

We have a database running let's go back to our code and start by fetching 2 github packages:

• go get -u gorm.io/gorm
• go get -u gorm.io/driver/postgres

Gorm is a ORM library to help us work with our database, I really enjoy it, is quite simple to use that's why I'm using it.

Let's create a database package inside of internal/ with a database.go file.

We are going to have an InitDatabase() func that will handle our database connection that will return an instance of the DB struct of our gorm package (**gorm.DB) or an error. Then we will be setting up multiple variables to build our connection string for PostgreSQL. It's not good practice to hard code these values so I'll be using the **os* package to get environment variables that I'll be then exporting locally in my terminal for now.

Once I have my connection string, I'll call the Open() method of gorm and open my connection string with the postgres driver of gorm.

Let's head back to cmd/server/main.go and call InitDatabase() in our Run() func.

Before run our API don't forget to export the variable we have defined in database.go in the terminal window where you'll be running the program.

• export DB_USERNAME=postgres
• export DB_PASSWORD=postgres
• export DB_HOST=localhost
• export DB_TABLE=postgres
• export DB_PORT=5432

Let's run go run cmd/server/main.go and check that InitDatabase() did not send an error.

Looks Good ? . Now it is time ti implement our student service.

The Service

Time to implement our student service and we are going to head back to the internal/ directory and add a subdirectory services/ and inside of it let's create a package student containing a student.go file.

In student.go we are going to implement two structs, a Service struct and Student struct. The Service struct will define our service that will contain only a field DB typed as a reference to our **gorm.DB* so we can pass in our database when we instantiate it.
The Student will define an actual student and we will pass in an additional parameter or field of gorm.Model so our ORM know the fields to implement in our database for a student.

We are also going to create a NewService() func with an argument db to pass in our database from cmd/server/main.go later. This func will return an instance of our service.

Okay, we have this but something is missing, an interface. We need to implement a StudentService interface that will tells what methods we need to implement in our service to be of type StudentService, it is a contract we have to respect. So let's write the methods we need to have a StudentService.

Now our StudentService is defined, we just need to implement all of this methods. They will be all receivers functions of Service so we can call them later to be assigned to our endpoints. Let's implement them!

Since they are receivers of Service we have access to the DB field we wrote earlier. And remember that DB is actually of type **gorm.DB* so we have access to their methods and we can make our SQL queries easily ?

For example for GetAllStudents() we can just use the Find() method from gorm and write the results of it to a temporary variable students and then return it.

Let's implement the rest of our methods! I'll be pushing the actual code to a GitHub repository if you want to get it there ??

Perfect, now we can go back to our internal/http/handler.go file and use our student service.

HTTP Endpoints Methods & JSON

In our handler file we can now create the rest of our endpoints as we did for api/status but this time the second argument will be functions we will create and those functions will be calling our services methods that we have created previously. And for that, we are going to add a field Service to our Handler struct and that field will be typed as a pointer to the Service of the student package. Also we will pass this service in our NewHandler() func so we can have access to it when we insatiate the handler.

⚠️ Attention! Rename the package from handler to http.

Let's continue by adding our endpoints to InitRoutes() and then defining the handler functions that will call our service functions.

Okay, now let's write the body of this handlers functions and return a JSON response since this is a REST API. And here we will see that the gorilla/mux router will save us time for example to retrieve de id of our endpoints.

For example, in GetStudentById we can mux.Vars() passing in our request "r" and then get whatever parameter we passed in curly braces in our endpoint, in this case the {id}.
**Once we have the id we can parsed it from string to int or uint in this case and then call our Service method and fetched the student from database.
If there is no error we then can create a new **json encoder passing in our response writer "w" and encode the result (student) from our service.

Another thing I want to do before implement to rest of our handler functions is to improve the error handling in this function. We are just printing an error message but this is a REST API it is good practice to actually return an HTTP status code if we get an error.

For that I'll create an helper function named respondWithError() and I will also create a Response struct that will be useful. I'll be writing this in a new file helper.go inside of our package http.

⚠️ By the way I have just noticed that I've made a mistake!! I've been calling my package handler instead of http. Sorry for that just rename the package to http.

I fixed my package name ??

In respondWithError I write an header with a Status Error and encode the struct Response to json passing in a custom message with my error.

We can use this back in our handler.go

Also, we are going to set in our response header a Content-Type of application/json plus UTF-8 charset encoding and write an an HTTP status OK that will return 200 as code. If nothing goes wrong we don't enter in respondWithError and the status stays 200 ✅ .

Okay, GetStudentById is done now let's implement the rest of our functions.
Once again the source code for this is in my GitHub repository ?

Okay, moving on let's fix one thing in cmd/server/main.go before testing this. Let's import our StudentService in our main.go file and call NewService and passing it our database.InitDatabase() as argument. Then we just have to pass studentService to the NewHandler() call.

This reminds me, I want to handle database migrations.

Database migrations

Since we are using gorm this task is quite simple, there is a method in our ORM library that auto migrates the database when we init our database connection. Let's create a new file migrate.go inside of our package database in the internal/ directory.
There we will simply write an helper function MigrateDB that will migrate our database. We will call these function in our Run() in cmd/server/main.go.

Okay, I think we are ready to test our endpoints, finally !

API endpoints with Insomnia

To test our endpoints I'll use a software named Insomnia, you can also use Postman that is more famous I guess, I just enjoy using Insomnia but it is the same thing ?

Let's run our app with go run cmd/server/main.go

I've created my endpoints in Insomnia and I've posted a new Student and I don't get an error and I see that my preview is actually in JSON.

Let's test to get this student by it's school name "Havard" and then it's ID.

Let's add another student to test GET all students.

You can test the rest of the endpoints ? but it seems to be working just fine.

Write some tests

A good API is a well tested API, so I'll try to implement E2E Testing for that I'll be using the resty library that will allow us to test our http endpoints and the testify/assert library that will give us access to assert methods that will check if the result is equal to what we are expecting basically. Let's get this packages.

go get -u github.com/go-resty/resty/v2

go get -u github.com/stretchr/testify/assert

Then create a package test in our root directory and add to it a student_test.go file.
In this file we will create a constant with our BASE_URL so we don't have to write in all of our file. Then we will create a TestGetAllStudents() func. There we are going to call a new "client" to simulate our user, this client is an instance of resty.New() method.
Then we will call the R() method for request and make a GET() request passing in our endpoint url.

We assign a resp and err variable to it and if we got a error we call t.Fail().
If we have no error then we assert.Equal() comparing the status code of our resp variable (result of the get request to out endpoint) to a 200 status code.

And this is basically it!

Now if we restart our app ( go run cmd/server/main.go ) and we cd into the testing package and run go test we can see that our test has passed.

Now to have some test coverage we just need to test the rest of our endpoints. For example, the TestPostStudent() would look like this

Okay, let's move on we are almost done ?

Let's Dockerize our API

Our API is functional but every time we want to use it with our database we need to run that long command in our terminal

docker run ----name students-db ----env POSTGRES_PASSWORD=postgres -p 5432:5432 -d postgres

And then we have to export our env variables, and still hit the go run to actually run our app. There is a way to simplify this and it is actually good if we want to deploy this API one day. That solution is to add Docker to our app, we will create a Dockerfile that will be in charge of building our API and then a docker-compose.yml file that will be in charge of creating multiple services (containers) and connecting them together. We will have 2 containers or services as you want to call it, one of them will be our PostgresSQL database and the other our API that will be buid with the help of our Dockerfile.

Let's start by creating a Dockerfile in our root directory. In there we will just be fetching a golang image from the docker hub, then create a workspace directory inside of it, add the content of our current directory (so all of our app), build our executable and then run it.

Now let's add the docker-compose.yml file also in our root directory. As I said before in there I have 2 services db and api, one based on a PostgreSQL image and another from our Dockerfile, then we map the right ports in both containers. I have a volume from my db so the database content is synced in a local folder in my machine.
Both services are in the same network so they can communicate.
The api service depends on the db service meaning it cannot start if the db does not work. And then I finally set the default bridge network as a link layer.

To test this out you need to stop the previous database container we were running. Remember check with docker ps if you have it running and then run a docker stop <CONTAINER ID> or docker stop <CONTAINER_NAME> .

Now all we have to do is run docker-compose upto start our API, and docker-compose down to stop it ?

Attention: The first time running docker-compose or whenever you want to rebuild the entire thing use docker-compose up --build

We are reaching the end, let's just check if our endpoints are still working.

Seems to be good!
Okay, our app is Dockerized , it is a lot simpler to start our api using Docker ??

Conclusion ?

Okay we are officially done, I know it has been a bit long but I am glad I wrote this article because I've learned a lot also by trying to explain this to you guys.

Don't hesitate to check my Youtube Channel or my Twitter account and contact me if you have any questions or remarks ?.
Also the source code if you need to check it.

Youtube Channel

Source Code

See you soon! ??

This content originally appeared on DEV Community and was authored by Lucas Neves Pereira