This content originally appeared on DEV Community and was authored by Jack Kweyunga
Introduction
The world of technology is moving quickly. It's moving so fast that keeping up with everything new can be overwhelming. Therefore, the need to automate some basic tasks is inevitable.
Most development jobs today require you to deploy the application you've built (API, chat-bot, web app, etc.) either as a final stage or during development to show real-time progress to your clients. As a developer, you need to understand the deployment process. However, your main focus should be on building the product and its logic.
To save time and stay focused, you should automate the deployment process before starting development. Once this is set up, you can concentrate on building the product.
Automating deployments is part of DevOps practices that improve an organization's efficiency. It reduces lead time, allowing changes to be shipped faster (in minutes), decreases deployment time since the process is automated, and enhances the overall work experience. This is known as Continuous Delivery, meaning released code is shipped to production faster and automatically. While there's more to the terminology, today we will learn how to automate deployments.
Follow up with this GitHub repository: https://github.com/jackkweyunga/auto-deploy-flask-to-server
Tools
Most of you have already heard of or used Ansible and GitHub Actions since they are popular tools. If you haven't, don't worry—they are simple and easy to understand. By the end of this article, you'll have a good start.
ANSIBLE is a general-purpose IT automation tool created using Python with a large community around it. It is mostly used for configuration management, provisioning resources, orchestration, and more. Read more
GITHUB ACTIONS is a feature by GitHub that lets you automate workflows such as running tests, builds, and releases based on given triggers. Read more
Setup
To accomplish our goal today, we need to package our application so that it's portable and easy to run. This is where Docker comes in. You have likely heard of Docker, and maybe even used it.
For this article, we'll be building and deploying a simple Flask API.
The folder structure will be as follows:
auto-deploy-flask-to-server
.github\
workflows\
docker-image.yml
deploy.yml
ansible\
flask-api\
files\
docker-compose.yml
tasks\
main.yml
deploy.yml
create-sudo-password-ansible-secret.sh
Dockerfile
app.py
requirements.txt
docker-compose.dev.yml
.dockerignore
.gitignore
Application ( Flask API )
In our app.py, there is a simple Flask route that returns a JSON object with the current time.
app.py
from flask import Flask
from datetime import datetime as dt
import os
app = Flask(__name__)
@app.route("/")
def heartbeat():
return {
"now": dt.utcnow()
}
if __name__ == "__main__":
# Get the DEBUG environment variable, defaulting to True if not set.
debug = os.environ.get("DEBUG", "True").lower() in ("true", "1")
app.run(debug=debug)
Docker
This is what the Dockerfile should look like.
Dockerfile
# Use an official Python runtime as a parent image
FROM python:3.11-slim
# Set environment variables
ENV PYTHONDONTWRITEBYTECODE=1 \
PYTHONUNBUFFERED=1
# Create and set the working directory
WORKDIR /app
# Copy the requirements file and install dependencies
COPY requirements.txt /app/
RUN pip install --no-cache-dir -r requirements.txt
# Copy the application code to the working directory
COPY . /app/
# Expose the port the app runs on
EXPOSE 5000
# Set the environment variable to ensure Python runs in production mode
ENV FLASK_ENV=production
# Command to run the application
CMD ["python", "app.py"]
After adding a Dockerfile, we can proceed to test our application image. Use the following docker command to build the image. Solve any occurring errors until the build is successful and the docker image runs correctly.
# Build docker image
# windows
docker build -t flask_app .
# linux
sudo docker build -t flask_app .
# Run the image
# windows
docker run -rm -p "5000:5000" flask_app
# linux
sudo docker run -rm -p "5000:5000" flask_app
Now let's add content to the docker-image.yml file in the .github/workflows directory. This will automate Docker image builds for our application. For more details about this file, please refer to this article: Upload Docker Images to GitHub: A Simple Guide with GitHub Actions. This setup will create a Docker image tagged with the commit hash and git branch, whenever we commit our code to GitHub.
.github/workflows/docker-image.yml
name: ci # Put the name of your choice
on:
workflow_dispatch:
push:
branches:
- "main" # Change to the branch name you are targeting
env:
REGISTRY: ghcr.io
IMAGE_NAME: ${{ github.repository }} # takes the name of the repository.
jobs:
build-publish-deploy:
name: build and push docker
runs-on: ubuntu-latest
permissions:
contents: write
packages: write
steps:
- name: checkout
uses: actions/checkout@v3
- name: Set up Docker Builds
uses: docker/setup-buildx-action@v2
- name: Login to Container registry
uses: docker/login-action@v2
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action@69f6fc9d46f2f8bf0d5491e4aabe0bb8c6a4678a
with:
images: |
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
tags: |
type=ref,event=branch
type=ref,event=tag
type=ref,event=pr
type=sha
flavor: |
latest=auto
prefix=
suffix=
- name: Build and push hash tagged image
uses: docker/build-push-action@v2
with:
context: .
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
cache-from: type=registry,ref=${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:latest
cache-to: type=inline
Let us add content to the docker-compose.dev.yml file in the root directory of our project. Use this docker-compose file for testing and development if needed.
docker-compose.dev.yml
services:
web:
build: .
container_name: flask_app
environment:
- FLASK_ENV=development
- DEBUG=True
ports:
- "5000:5000"
restart: always
logging:
driver: "json-file"
options:
max-size: "10m"
max-file: "3"
deploy:
resources:
limits:
memory: "256m"
cpus: "0.50"
To run the docker-compose file, run te following commands.
# windows
docker compose -f docker-compose.dev.yml build
docker compose -f docker-compose.dev.yml up -d
# linux
sudo docker compose -f docker-compose.dev.yml build
sudo doccker compose -f docker-compose.dev.yml up -d
Ansible
It's time to finish setting up Ansible in our project. We will be making changes to the contents of the Ansible folder.
ansible\
flask-api\
files\
docker-compose.yml
tasks\
main.yml
deploy.yml
create-sudo-password-ansible-secret.sh
We've organized ansible in roles. flask-api is an ansible role with all the files and tasks needed to deploy our flask application. But hey, what's an ansible role ?
An ansible role provides a structured way to organize files, vars, templates and tasks related to a given automation.
You can always add another role for other automation. For example: A database role
Let's add content to the docker-compose.yml file in the ansible/flask-api/files directory, which is our production Docker Compose file. Notice that we no longer set build: .; instead, we use the remote path to the Docker image we built earlier.
You might ask yourself why this Docker Compose file look similar to the one in the root directory. Well, this is because your testing and, if possible, development environments should be as close as possible to the production environment. This helps minimize errors and bugs that might occur in production but are not detectable in testing or development. It doesn't eliminate them entirely but reduces them.
Additional services are portainer and watchtower.
Portainer is a web interface for Docker that helps you manage containers, images, stacks, and more.
Watchtower is crucial in our setup because it pulls new images after they are published with GitHub Actions. Since we are going to be using ghcr.io , in a public way, we do not need to setup any docker authentication for it. Otherwise, we had to setup docker authentication for private container registries.
ansible/flask-api/files/docker-compose.yml
volumes:
portainer-data:
services:
portainer:
image: portainer/portainer-ce:alpine
container_name: portainer
command: -H unix:///var/run/docker.sock
ports:
- "9000:9000"
volumes:
# Connect docker socket to portainer
- "/var/run/docker.sock:/var/run/docker.sock"
# Persist portainer data
- "portainer_data:/data"
restart: always
watchtower:
container_name: "watchtower"
image: "docker.io/containrrr/watchtower"
volumes:
- /var/run/docker.sock:/var/run/docker.sock
# To enable docker authentication, uncomment the line below.
# You also need to make sure you are logged in to docker in the server
# E.g by running: sudo docker login ghcr.io
# - /root/.docker/config.json:/config.json:ro
restart: always
environment:
TZ: Africa/Dar_es_Salaam
WATCHTOWER_LIFECYCLE_HOOKS: "1" # Enable pre/post-update scripts
command: --debug --cleanup --interval 30
web:
image: ghcr.io/jackkweyunga/auto-deploy-flask-to-server:main
environment:
- FLASK_ENV=production
- DEBUG=${DEBUG}
ports:
- "5000:5000"
restart: always
logging:
driver: "json-file"
options:
max-size: "10m"
max-file: "3"
deploy:
resources:
limits:
memory: "256m"
cpus: "0.50"
Let's define our task in ansible/flask-api/tasks/main.yml. What does it do?
Creates a project directory on the remote server if it doesn't exist. In our case, it's the
.flask-apifolder.Copies relevant files to the server. In our case, it's the
docker-compose.ymlfile.Runs the compose commands to pull and run the services.
ansible/flask-api/tasks/main.yml
# main task
- name: Deploy app
become: true
block:
- name: Create a directory if it does not exist
ansible.builtin.file:
path: "{{ lookup('ansible.builtin.env', 'HOME_DIR') }}/{{ lookup('ansible.builtin.env', 'PROJECT_DIR') }}/.flask-app"
state: directory
- name: Copy compose file to server
ansible.builtin.copy:
src: "files/"
dest: "{{ lookup('ansible.builtin.env', 'HOME_DIR') }}/{{ lookup('ansible.builtin.env', 'PROJECT_DIR') }}/.flask-app/"
# Uncomment this when working with private repositories
# Make sure the environment variables; DOCKER_TOKEN and DOCKER_USERNAME are provided
#
# - name: Login to Docker via vars/main.yaml
# shell: "echo \"{{ lookup('ansible.builtin.env', 'DOCKER_TOKEN') }}\" | docker login ghcr.io -u {{ lookup('ansible.builtin.env', 'DOCKER_USERNAME') }} --password-stdin"
- name: Docker Compose Up
community.docker.docker_compose_v2:
project_src: "{{ lookup('ansible.builtin.env', 'HOME_DIR') }}/{{ lookup('ansible.builtin.env', 'PROJECT_DIR') }}/.flask-app"
pull: "always"
register: output
- name: Debug output
ansible.builtin.debug:
var: output
For Ansible to run automation tasks, you need to define a playbook. But what is a playbook?
A playbook in Ansible contains a list of tasks and/or roles to be executed against a list of hosts (e.g., web servers).
In our setup, we define the playbook ansible/deploy.yml . What does it do?
Provides a way to pass sudo password when running privileged
sudocommandsDefines the roles to be run
Defines the remote environment variables ( We do not need for this setup )
ansible/deploy.yml
---
- hosts: webservers
# an encrypted ansible secret file containing the sudo password
vars_files:
- secret
roles:
- services
environment:
DEBUG: "{{ lookup('ansible.builtin.env', 'DEBUG') }}"
To ensure that the encrypted sudo password is available ( secret ) , we need to run the script create-sudo-password-ansible-secret.sh . What does the script do ?
creates the vault password file (
vault.txt)create the encrypted vars file (
secret) containing the ansible_sudo_pass which will be passed when running privileged sudo commands with ansible.
ansible/create-sudo-password-ansible-secret.sh
#!/bin/bash
# variables
VAULT_PASSWORD=$(openssl rand -base64 12)
VAULT_PASSWORD_FILE="ansible/vault.txt"
VAULT_FILE="ansible/secret"
SUDO_PASSWORD="$1"
SUDO_PASSWORD_FILE="/tmp/sudo-password"
# sudo passord is required
if [ -z "${SUDO_PASSWORD}" ]; then
echo "Usage: $0 <sudo-password>"
exit 1
fi
# create vault password file
echo "${VAULT_PASSWORD}" > "${VAULT_PASSWORD_FILE}"
# create a sudo password file
echo "ansible_sudo_pass: ${SUDO_PASSWORD}" > "${SUDO_PASSWORD_FILE}"
# encrypt sudo password
ansible-vault encrypt --vault-password-file "${VAULT_PASSWORD_FILE}" "${SUDO_PASSWORD_FILE}" --output "${VAULT_FILE}"
That's it for Ansible. Coming up next: GitHub Actions setup.
GitHub Actions
We previously created a GitHub Actions workflow file .github/workflows/docker-image.yml that builds and publishes a Docker image for us on push events.
In this section, we are going to add another workflow, that will run the deploy.yml ansible playbook. Let's add content to the file: .github/workflows/ansible-deploy.yml
.github/workflows/ansible-deploy.yml
name: ansible-deploy
on:
workflow_dispatch:
inputs:
REMOTE_USER:
type: string
description: 'Remote User'
required: true
default: 'ubuntu'
HOME_DIR:
type: string
description: 'Home Directory'
required: true
default: '/home/ubuntu'
TARGET_HOST:
description: 'Target Host'
required: true
default: "example.com" # Change this to your server IP or Domain
jobs:
ansible:
runs-on: ubuntu-latest
env:
DEBUG: 0
steps:
- name: Checkout
uses: actions/checkout@v2
- name: Add SSH Keys
run: |
cat << EOF > ansible/devops-key
${{ secrets.SSH_DEVOPS_KEY_PRIVATE }}
EOF
- name: Update devops private key permissions
run: |
chmod 400 ansible/devops-key
- name: Install Ansible
run: |
pip install ansible
- name: Adding or Override Ansible inventory File
run: |
cat << EOF > ansible/inventory.ini
[webservers]
${{ inputs.TARGET_HOST }}
EOF
- name: Adding or Override Ansible Config File
run: |
cat << EOF > ./ansible/ansible.cfg
[defaults]
ansible_python_interpreter='/usr/bin/python3'
deprecation_warnings=False
inventory=./inventory.ini
remote_tmp="${{ inputs.HOME_DIR }}/.ansible/tmp"
remote_user="${{ inputs.REMOTE_USER }}"
host_key_checking=False
private_key_file = ./devops-key
retries=2
EOF
- name: Run deploy playbook
run: |
sh ansible/create-sudo-password-ansible-secret.sh ${{ secrets.SUDO_PASSWORD }}
ANSIBLE_CONFIG=ansible/ansible.cfg ansible-playbook ansible/deploy.yml --vault-password-file=ansible/vault.txt
The workflow installs Ansible on the GitHub Ubuntu runner.
It creates the Ansible inventory file
ansible/inventory.ini, filling it with the target host IP or domain.It creates the Ansible configuration file
ansible/ansible.cfgbased on the GitHub Actions inputs and secrets.It runs the script to create the sudo password Ansible secret.
It runs the Ansible playbook.
Notice areas in the YAML file where values are read from the variable secret. This refers to the GitHub Actions secrets. You can edit them by navigating to settings/secrets/actions in your GitHub repository. For this workflow, we need to add the following GitHub secrets:
SUDO_PASSWORD: The password of the remote user
SSH_DEVOPS_KEY_PRIVATE: The SSH private key with access to the remote server
Use GitHub secrets to store private information needed by a workflow. That's it for GitHub Actions. Now, let's review what we have done so far.
Operation
Now that our setup is complete, let's learn how it operates.
When you push to the main branch, a new Docker image will be published to the GitHub container registry. If the workflow fails, you need to check and fix the error until it works again.
The workflow
ansible-deployis manually triggered to configure and deploy the application to the server. This is done for the first time and whenever configurations change. There is no need to run it regularly.Whenever a new image is published, Watchtower will automatically pull the image into the server and continuously update the containers running the image.
In this way, we have completed the set up for automated deployments to a server using ansible and GitHub actions.
Conclusion
Wow! It's been quite a journey. If you've made it this far, you've probably learned a thing or two about automated deployments with Ansible and GitHub Actions. If anything is unclear, please ask, and I'll be happy to help. I hope you enjoyed the article.
Seeking expert guidance in Ops, DevOps, or DevSecOps? I provide customized consultancy services for personal projects, small teams, and organizations. Whether you require assistance in optimizing operations, improving your CI/CD pipelines, or implementing strong security practices, I am here to support you. Let's collaborate to elevate your projects. Contact me today.
Whats next ?
One can go further and do the following to improve or scale up the process:
Add automated tests that run before an image is built and published.
Add Ansible configuration tests that run when Ansible configurations are updated.
Scale up the setup with Kubernetes and Argo CD (depending on the organization's needs and components).
This content originally appeared on DEV Community and was authored by Jack Kweyunga
Jack Kweyunga | Sciencx (2024-09-06T13:20:40+00:00) How to Automate Deployment with Ansible and GitHub Actions. Retrieved from https://www.scien.cx/2024/09/06/how-to-automate-deployment-with-ansible-and-github-actions/
Please log in to upload a file.
There are no updates yet.
Click the Upload button above to add an update.