This content originally appeared on DEV Community and was authored by Lorenzo Garuti
In this tutorial we are going to see how to use Nginx as an ingress controller for our Docker swarm cluster.
Prerequisites
For this tutorial we only need a running Docker swarm cluster. You can setup a docker swam cluster in a few setp.
Nginx ingress controller
The image that we will use is based on the excellent work of foxylion then revisited on my docker swarm ingress image.
The image it’s very simple, the magic part is the ingress.py file so let’s take a look on how the magic happens.
The image needs to be deployed on a manager node, when the image starts the docker-entrypoint.sh is called:
openssl req -new -newkey rsa:4096 -days 365 -nodes -x509 \ -subj "/C=IT/ST=Denial/L=Italy/O=IT/CN=dummy.cert.io" \ -keyout /etc/nginx/default.key -out /etc/nginx/default.crtpython ingress.py &
echo $! > /ingress/ingress.pid
exec nginx -g "daemon off;"
In the above snippet we see the main part of the docker-entrypoint.sh.
In our container there are now two process:
- the ingress.py “daemon”
- and the nginx process
The first line of our snippet create a dummy ssl certificate for nginx, we will cover this in the “SSL mode in depth”.
The ingress.py daemon then will check all the services on the docker swarm cluster. If a service has a label named “ingress.host” the nginx configuration will be regenerated based on the value of the labels.
Here is a list of valid labels:
- ingress.host: the nginx server name (virtualhost). Example: my-service.company.tld
- ingress.port: the port which serves the service in the cluster.
- ingress.virtual_proto: the protocol used to connect to the backends
- ingress.ssl: specify enable to enable ssl-assthrough. More detail on ssl in the “SSL mode in depth”
- ingress.ssl_redirect: specify enable to enable http to https redirect
There are also some environment variales that we can specify on our nginx ingress controller container. Here the list of accepted environment variables:
- DOCKER_HOST: the connect string where docker-py library tries to connect to the docker daemon
- UPDATE_INTERVAL: the time in seconds that ingress.py wait before checking for new services in the docker swarm cluster. Default 30 seconds.
- Debug: enable or disable debug mode
- USE_REQUEST_ID: enable or disable Request-Id header
- LOG_FORMAT: specify log format, valid values are json, custom or default
- LOG_CUSTOM: specify the nginx log format
- PROXY_MODE: define nginx SSL proxy mode. Valid values are ssl-passthrough (default) or an used defined value to enable SSL bridging or SSL termination mode. More detail on ssl in the “SSL mode in depth”
SSL mode in depth
SSL Passthrough (default)
SSL passthrough passes HTTPS traffic to a backend server without decrypting the traffic on the load balancer. The data passes through fully encrypted, which precludes any layer 7 actions.
SSL Termination/Offloloading
SSL termination (a.k.a. SSL Offloading) decrypts all HTTPS traffics when it arrives at the load balancer (our docker swarm ingress controller), and the data is sent to the destination server as plain HTTP traffic (our http backend deployment)
SSL Bridging
Opposed to SSL termination the traffic from the load balancer and the destination is not in plain HTTP traffic but the traffic is encrypted again.
Setup the stack
We are now ready to deploy our nginx ingress controller, in one docker swarm manager node download my docker swarm ingress repository:
git clone https://github.com/garutilorenzo/docker-swarm-ingress.git
we need to create an overlay network first. This network will then be attached to our backends. Create the network with:
docker network create --driver overlay ingress-routing
then deploy the stack:
cd docker-swarm-ingress/
docker stack deploy -c examples/docker-ingress-stack.yml nginx-ingress
we can check the status of our stack using docker stack ps:
docker stack ps nginx-ingress
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
x47oswdzbnlp nginx-ingress_nginx.1 garutilorenzo/docker-swarm-ingress:latest node-2 Running Running 12 seconds ago
we can also check if our nginx ingress controller is working correctly using curl or a web browser.
We are now ready to deploy our first backend service, to do this we can use an example service:
docker stack deploy -c examples/example-service.yml example-http
if we check the nginx ingress controller log we can see that the nginx server has restarted:
docker service logs -f --tail=100 nginx-ingress_nginx
nginx-ingress_nginx.1.njwey0rlfvdw@node-2 | 2021/09/30 14:05:12 [notice] 11#11: exiting
nginx-ingress_nginx.1.njwey0rlfvdw@node-2 | 2021/09/30 14:05:12 [notice] 11#11: exitnginx-ingress_nginx.1.njwey0rlfvdw@node-2 | 2021/09/30 14:05:42 [notice] 41#41: signal process started
and now we can try to reach my-service.company.tld (update your DNS or your host file and set my-service.company.tld pointing to one of your docker swarm cluster ip’s)
HTTPS, SSL passthrough mode
We can now try to expose my-service.company.tld using https with SSL passthrough mode.
First we need to delete our example-http stack:
docker stack rm example-http
then we need to create our self-signed ssl certificates and we need also to create the nginx secrets:
mkdir certs && sudo openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout ./certs/nginx.key -out ./certs/nginx.crtGenerating a RSA private key
...................................................................................................+++++
.....................+++++
writing new private key to './certs/nginx.key'
-----
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [AU]:IT
State or Province Name (full name) [Some-State]:Italy
Locality Name (eg, city) []:
Organization Name (eg, company) [Internet Widgits Pty Ltd]:
Organizational Unit Name (eg, section) []:
Common Name (e.g. server FQDN or YOUR name) []:my-service.company.tld
Email Address []:me@company.tlddocker secret create nginx_cert certs/nginx.crt
docker secret create nginx_key certs/nginx.key
now we can deploy our ssl backends:
docker stack deploy -c examples/example-ssl-service.yml example-httpsCreating config example-https_nginx_config
Creating config example-https_nginx_options
Creating config example-https_nginx_dhparams
Creating service example-https_nginx
check the nginx ingress controller logs and when the nginx daemon is restarted try to reach my-service.company.tld:
we are now exposing our example-https using SSL passthrough.
HTTPS, SSL Termination/Offloloading
We can now test SSL Termination/Offloading mode, we need to delte both the stack:
docker stack rm example-https
Removing service example-https_nginx
Removing config example-https_nginx_dhparams
Removing config example-https_nginx_options
Removing config example-https_nginx_config
docker stack rm nginx-ingress
Removing service nginx-ingress_nginx
Now we need to create the ssl certificates for our domain, exposed directly by the nginx ingress controller:
sudo openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout ./certs/my-service.key -out ./certs/my-service.crt
Generating a RSA private key
......+++++
...............................................................+++++
writing new private key to './certs/my-service.key'
----------
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
----------
Country Name (2 letter code) [AU]:IT
State or Province Name (full name) [Some-State]:Italy
Locality Name (eg, city) []:
Organization Name (eg, company) [Internet Widgits Pty Ltd]:
Organizational Unit Name (eg, section) []:
Common Name (e.g. server FQDN or YOUR name) []:my-service.company.tlddocker secret create my-service.company.tld.key certs/my-service.key
docker secret create my-service.company.tld.crt certs/my-service.crt
Now we can deploy our nginx ingress controller in SSL Termination/Offloloading mode.
docker stack deploy -c examples/docker-ingress-stack-ssl_term_bridg.yml nginx-ingress
and we can also deploy our http example backend:
docker stack deploy -c examples/example-service-ssl-termination.yml example-https-termination
check the nginx ingress controller logs and when the nginx daemon is restarted try to reach my-service.company.tld. We are now reacing my-service.company.tld in https mode but unlike the SSL passthrough mode the ssl certificates are served directly by our nginx ingress controller.
HTTPS, SSL Bridging
For the SSL bridging test we need to delete only our backend service “example-https-termination”. The configuration of the nginx ingress controller for SSL bridging and SSL Termination/Offloloading are the same.
docker stack rm example-https-terminationdocker stack deploy -c examples/example-service-ssl-bridging.yml example-htts-bridging
check the nginx ingress controller logs and when the nginx daemon is restarted try to reach my-service.company.tld. We are now reacing my-service.company.tld in https mode the ssl certificates are served directly by our nginx ingress controller and the communication between the nginx ingress controller and our backend service “example-htts-bridging” is encrypted too.
Nginx tips
For the SSL mode nginx is acting as a layer 7 and layer 4 porxy, the nginx configurations to achieve this is the following:
stream {
map $ssl_preread_server_name $name {
my-service.company.tld backend-example-https_nginx;
}
# my-service.company.tld - ywy8bs1b6h8td118mkjrr70eq - HTTPS Passthrough
upstream backend-example-https_nginx {
server example-https_nginx:443;
}
proxy_protocol on;
server {
listen 443;
proxy_pass $name;
ssl_preread on;
}
}
and in the backend configuration we need to add this to the nginx config:
listen 443 ssl proxy_protocol;
set_real_ip_from nginx-ingress_nginx; # <- name of our nginx ingress controller service
real_ip_header proxy_protocol;
This content originally appeared on DEV Community and was authored by Lorenzo Garuti
Lorenzo Garuti | Sciencx (2021-09-30T15:46:42+00:00) Docker swarm nginx ingress controller. Retrieved from https://www.scien.cx/2021/09/30/docker-swarm-nginx-ingress-controller/
Please log in to upload a file.
There are no updates yet.
Click the Upload button above to add an update.