DevOps Monitoring and Automation Tool using Jenkins, Prometheus, Grafana and Docker

In this guide, we will walk through the process of setting up a DevOps monitoring and automation tool and understanding how different tools can be used together for this purpose: Jenkins for continuous integration and continuous deployment(CI/CD), Prom…


This content originally appeared on DEV Community and was authored by Nigel Bomett

In this guide, we will walk through the process of setting up a DevOps monitoring and automation tool and understanding how different tools can be used together for this purpose: Jenkins for continuous integration and continuous deployment(CI/CD), Prometheus for metrics collection, Grafana for visualization, and Docker for containerization.

...

Install Docker

If you haven't already installed Docker, follow the installation instructions for your operating system from the Docker website.

Set up Jenkins

Run Jenkins in a Docker Container:

 ```
 docker run -u 0 --privileged --name jenkins -it -d -p 8080:8080 -p 
 50000:50000 
 \-v/var/run/docker.sock:/var/run/docker.sock 
 \-v $(which docker):/usr/bin/docker 
 \-v /home/jenkins_home:/var/jenkins_home 
 \jenkins/jenkins:latest
```

Access and Setup Jenkins:

  • Open your browser and go to http://localhost:8080/

  • You will be requested to provide a password and there is a guide
    on where to find it however, in case you can not locate the file
    you can run the command: docker logs <docker container id> where
    the 'docker container id' is the id of the docker container running
    Jenkins, the password will be displayed in the logs.

  • For this tutorial, you can select to install suggested plugins.
    We will install any other required plugins later on.

  • From the dashboard add a new item, I will be using, Pipeline.

Install Plugins:

  • Go to "Manage Jenkins" > "Manage Plugins".

  • In the "Available" tab, search for "Docker Plugin", "Docker Pipeline
    Plugin", "Prometheus metrics Plugin" and install them. Restart Jenkins
    if prompted.

Set Up Prometheus and Grafana

  • Create a Docker Network:
    docker network create monitoring

  • Run Prometheus:
    Create a prometheus.yml configuration file:

global:
  scrape_interval: 15s

scrape_configs:
  - job_name: 'prometheus'
    static_configs:
      - targets: ['localhost:9090']

  - job_name: 'jenkins'
    static_configs:
      - targets: ['jenkins:8080']
      metrics_path: '/prometheus'

  - job_name: 'node'
    static_configs:
      - targets: ['node-exporter:9100']

  • Run Prometheus with this configuration:
docker run -d --name=prometheus 
\--network=monitoring 
\-p 9090:9090 
\-v $(pwd)/prometheus.yml:/etc/prometheus/prometheus.yml \prom/prometheus

  • Run Node Exporter:
docker run -d --name=node-exporter 
\--network=monitoring 
\--restart=unless-stopped 
\prom/node-exporter

  • Run Grafana:
docker run -d --name=grafana 
\--network=monitoring 
\-p 3000:3000 
\grafana/grafana

Configure Grafana

  • Access Grafana
    Open your browser and go to http:localhost:3000.Log in with the default username and password(admin).

  • Add Prometheus as a Data Source:
    Go to "Data Sources" on the left sidebar, "Add data source" and select "Prometheus".
    Enter http://prometheus:9090 as the Prometheus server URL.
    Click "Save & Test"

  • Create a Dashboard:
    Select "Dashboards" from the left sidebar to create a new dashboard. You can import dashboard using ID or URL. For example ID: '11159'
    You can edit one of the panels or 'add visualization', then add a query for the data you want to see.
    For this example, we will be looking at how long it takes Jenkins to process a job. Add the query:
    default_jenkins_builds_last_build_duration_milliseconds

Automate application deployment with Jenkins

  • Set Up a Jenkins Pipeline: From the application you had set up in Jenkins, go to "configure" and scroll down to "Pipeline Script" The script below will guide you, you may customize to fit the application you are deploying:
pipeline {
    agent any

    environment {
        DB_HOST = 'your-db-host'
        DB_USER = 'your-db-user'
        DB_PASS = 'your-db-password'
        DB_NAME = 'your-db-name'
    }

    stages {
        stage('Checkout') {
            steps {
                script {
                    checkout([$class: 'GitSCM', branches: [[name: '*/main']],
                      doGenerateSubmoduleConfigurations: false,
                      extensions: [[$class: 'CleanCheckout']],
                      submoduleCfg: [],
                      userRemoteConfigs: [[url: 'https://github.com/yourusername/your-nodejs-typescript-api-repo.git',
                      credentialsId: 'git-credentials-id']]
                    ])
                }
            }
        }

        stage('Install Dependencies') {
            steps {
                sh 'npm install'
            }
        }

        stage('Build') {
            steps {
                sh 'npm run build'
            }
        }

        stage('Test') {
            steps {
                sh 'npm test'
            }
        }

        stage('Build Docker Image') {
            steps {
                script {
                    sh 'docker build -t your-dockerhub-username/your-nodejs-typescript-api:latest .'
                }
            }
        }

        stage('Push Docker Image') {
            steps {
                script {
                    withCredentials([usernamePassword(credentialsId: 'dockerhub-credentials', usernameVariable: 'DOCKER_USERNAME', passwordVariable: 'DOCKER_PASSWORD')]) {
                        sh 'echo $DOCKER_PASSWORD | docker login -u $DOCKER_USERNAME --password-stdin'
                    }
                    sh 'docker push your-dockerhub-username/your-nodejs-typescript-api:latest'
                }
            }
        }

        stage('Deploy') {
            steps {
                script {
                    sh '''
                    docker run -d -p 3000:3000 \
                        -e NODE_ENV=$NODE_ENV \
                        -e DB_HOST=$DB_HOST \
                        -e DB_USER=$DB_USER \
                        -e DB_PASS=$DB_PASS \
                        -e DB_NAME=$DB_NAME \
                        your-dockerhub-username/your-nodejs-typescript-api:latest
                    '''
                }
            }
        }

        stage('Monitor') {
            steps {
                script {
                    // Additional monitoring setup can be added here
                }
            }
        }
    }

    post {
        success {
            echo 'Build succeeded!'
        }
        failure {
            echo 'Build failed!'
        }
    }
}


Credentials

If you will be pulling code from Github you will need to add credentials for git hub and also Docker.
Go to "Manage Jenkins" > "Credentials" select global, then add your credentials for both Github and Docker. Take note of the id as this is what is used as reference in the Pipeline Script.

Build Application

Build your application and view the metrics on the Grafana dashboard 😃.

Image description


This content originally appeared on DEV Community and was authored by Nigel Bomett


Print Share Comment Cite Upload Translate Updates
APA

Nigel Bomett | Sciencx (2024-07-18T20:27:11+00:00) DevOps Monitoring and Automation Tool using Jenkins, Prometheus, Grafana and Docker. Retrieved from https://www.scien.cx/2024/07/18/devops-monitoring-and-automation-tool-using-jenkins-prometheus-grafana-and-docker/

MLA
" » DevOps Monitoring and Automation Tool using Jenkins, Prometheus, Grafana and Docker." Nigel Bomett | Sciencx - Thursday July 18, 2024, https://www.scien.cx/2024/07/18/devops-monitoring-and-automation-tool-using-jenkins-prometheus-grafana-and-docker/
HARVARD
Nigel Bomett | Sciencx Thursday July 18, 2024 » DevOps Monitoring and Automation Tool using Jenkins, Prometheus, Grafana and Docker., viewed ,<https://www.scien.cx/2024/07/18/devops-monitoring-and-automation-tool-using-jenkins-prometheus-grafana-and-docker/>
VANCOUVER
Nigel Bomett | Sciencx - » DevOps Monitoring and Automation Tool using Jenkins, Prometheus, Grafana and Docker. [Internet]. [Accessed ]. Available from: https://www.scien.cx/2024/07/18/devops-monitoring-and-automation-tool-using-jenkins-prometheus-grafana-and-docker/
CHICAGO
" » DevOps Monitoring and Automation Tool using Jenkins, Prometheus, Grafana and Docker." Nigel Bomett | Sciencx - Accessed . https://www.scien.cx/2024/07/18/devops-monitoring-and-automation-tool-using-jenkins-prometheus-grafana-and-docker/
IEEE
" » DevOps Monitoring and Automation Tool using Jenkins, Prometheus, Grafana and Docker." Nigel Bomett | Sciencx [Online]. Available: https://www.scien.cx/2024/07/18/devops-monitoring-and-automation-tool-using-jenkins-prometheus-grafana-and-docker/. [Accessed: ]
rf:citation
» DevOps Monitoring and Automation Tool using Jenkins, Prometheus, Grafana and Docker | Nigel Bomett | Sciencx | https://www.scien.cx/2024/07/18/devops-monitoring-and-automation-tool-using-jenkins-prometheus-grafana-and-docker/ |

Please log in to upload a file.




There are no updates yet.
Click the Upload button above to add an update.

You must be logged in to translate posts. Please log in or register.