Make slim Rails images

With deploying on Kubernetes in view, the production images aim to be small. It isn’t so easy to find documentation on how to produce slim Rails production images, so we present a solution here as well as a development hot-reload image.

TL;T…


This content originally appeared on DEV Community and was authored by NDREAN

With deploying on Kubernetes in view, the production images aim to be small. It isn't so easy to find documentation on how to produce slim Rails production images, so we present a solution here as well as a development hot-reload image.

TL;TR

A development mode Debian based image "slim-buster" (using the apt package manager) can be around 1000Mb. To go down to a slim size of approx 200 Mb image, you may:

  1. use a two-staged Dockerfile
  2. use a Linux Alpine base (with the apk package manager)
  3. use only Webpack and remove Sprockets (plus much shorter building time)
  4. bundle the production gems locally (as opposed to the "global" host gems) to the folder 'vendor/bundle'.
  5. Make the container stateless by removing the logs and using a Redis cache store.

We present also the development Dockerfile and how to use images in a context of micro-services to run a Rails monolith app with docker-compose.

Production mode

We have a two-step building process starting from a ruby:alpine image.

First stage

In the first stage, size is not of utmost importance: its main task is compile the gems and the static assets. We upload there the needed tools for:

  • Bundler to compile the declared gems and save them in a local folder "vendor/bundle"
  • Webpacker to compile the static assets to the "/public" folder.

Bundler

We want Bundler to compile the gems needed for the app (the "Gemfile.lock" file) into the app's code local subfolder "/vendor/bundle" to minimize the code. We thus set the environment variable:

BUNDLE_PATH='vendor/bundle'

This also indicates to Bundler where the gems are located, so it is repeated in the second stage.

We want to bundle only the production gems:

bundle config set --without 'development test'

Webpacker

We want to use a Webpack only version of Rails, without Sprockets.

> rails new <app-name> --skip-sprockets ...

In particular, this removes the sass-rails gem which besides saving space, reduces a lot of building time.

In this stage, we want Webpack to compile and minimize the static assets into the "/public" folder.

> RUN bundle exec rake assets:precompile

Stateless container

We also want to redirect the logs from the container to STDOUT and STDERR. They will be lost but can further be used by a log collector. Just declare the logger in the file "/config/application.rb" and set:

#.env
RAILS_LOG_TO_STDOUT=true

We also use the built-in Redis cache store. Redis will be setup as LRU with maxmemory 100mb(set in "redis.conf" circa line 566) and a maxmemory policy maxmemory-policy allkeys-lru. We will declare the gem hiredis and set:

config.cache_store = :redis_cache_store, { url: ENV['SIDEKIQ_REDIS'] }

Serving the static assets

There are different strategies: use a CDN or a reverse-proxy eg Nginx, or serving them with the default configured app-server Puma.
Since the app will most probably be running on Kubernetes, behind an Ingres controller, we will most probably serve the static assets from a CDN, on a separate port. Therefor, we want to use the app server - Puma here - to serve the static file, before migrating to Kubernetes. Since Rails by default won't serve static files in production mode, we set:

#.env
RAILS_SERVE_STATIC_FILES=true

Second stage

The second building stage will set a user to remove root privileges.
This stage will simply copy the app code, the compiled static files and the bundled gems from the host into the container.

The mandatory tzdata package is needed by the tzinfo-data gem when running on Windows.

Since we use Postgres, we still need the libpq package so Rails can communicate with Postgres.

The app Dockerfile 

#alpine.prod.Dockerfile

ARG RUBY_VERSION
FROM ruby:${RUBY_VERSION:-3.0.1-alpine} AS builder

ARG BUNDLER_VERSION
ARG NODE_ENV
ARG RAILS_ENV

RUN apk -U upgrade && apk add --no-cache \
   postgresql-dev nodejs yarn build-base tzdata

WORKDIR /app

COPY Gemfile Gemfile.lock package.json yarn.lock ./

ENV LANG=C.UTF-8 \
   BUNDLE_JOBS=4 \
   BUNDLE_RETRY=3 \
   BUNDLE_PATH='vendor/bundle'

RUN gem install bundler:${BUNDLER_VERSION} --no-document \
   && bundle config set --without 'development test' \
   && bundle install --quiet 

RUN yarn --check-files --silent --production

COPY . ./

RUN bundle exec rake assets:precompile

############################################################
FROM ruby:${RUBY_VERSION}

ARG RAILS_ENV
ARG NODE_ENV

RUN apk -U upgrade && apk add libpq netcat-openbsd tzdata\
   && rm -rf /var/cache/apk/*

# -D --disabled-password, don't assign a pwd, so cannot login
RUN adduser -D app-user
USER app-user

COPY --from=builder --chown=app-user /app /app

ENV RAILS_ENV=$RAILS_ENV \
   NODE_ENV=$NODE_ENV \
   RAILS_LOG_TO_STDOUT=true \
   RAILS_SERVE_STATIC_FILES=true \
   BUNDLE_PATH='vendor/bundle'

WORKDIR /app
RUN rm -rf node_modules

To build the image to further push it to a registry, we would run:

docker build --build-arg RUBY-VERSION=3.0.1-alpine --build-arg NODE-ENV=production --build-arg RAILS_ENV=production . -f alpine.prod.Dockerfile

Dev mode

In the development stage, we do not need a multi-stage build since hot-reload is our priority, not size. We use a separate Webpack service run with webpacker-dev-server and code bindings to accelerate the changes. Size is not the priority, but when using Alpine and Webpack only, the compilation runs faster. It is largely inspired by these guys but somehow simplified.

Alpine proposes at the time of writing Nodejs LTS 14, Postgres 13 and Yarn 1.22 packages by default, which is largely acceptable.

ARG RUBY_VERSION
FROM ruby:${RUBY_VERSION} AS builder

ARG BUNDLER_VERSION
ARG NODE_ENV
ARG RAILS_ENV

ENV RAILS_ENV=${RAILS_ENV} \
   NODE_ENV=${NODE_ENV} \
   BUNDLER_VERSION=${BUNDLER_VERSION}

RUN apk update && apk add --no-cache \
   build-base postgresql-dev nodejs yarn \
   tzdata netcat-openbsd \
   && rm -rf /var/cache/apk/*

WORKDIR /app

COPY Gemfile Gemfile.lock package.json yarn.lock ./

ENV LANG=C.UTF-8 \
   BUNDLE_JOBS=4 \
   BUNDLE_RETRY=3
# BUNDLE_PATH='vendor/bundle'
# <- to bundle only the gems needed from Gemfile into local folder /vendor/bundle

RUN gem install bundler:${BUNDLER_VERSION} --no-document \
   # && bundle config set --without 'development test' \
   && bundle install --quiet \
   && rm -rf /usr/local/bundle/cache/*.gem \
   && find /usr/local/bundle/gems/ -name "*.c" -delete \
   && find /usr/local/bundle/gems/ -name "*.o" -delete

RUN yarn --check-files --silent

COPY . ./

How to use this with a code example

We built a toy Rails monolith app. This simple app increments on button click a counter whose value is saved to a Postgres database and to a Redis database, and the click triggers background jobs with Sidekiq/Redis and workers.

This Webpack-only has some React installed:

> bundle exec rails wepacker:install:react

Local dev mode [branch "master"]

Since we use Sidekiq, Redis and Postgres, you will need a running Redis server and Postgres service when running locally. You then launch a Sidekiq service (with it's Redis service), the Rails server and finally run Webpack is dev mode. We used the process manager Overmind. Once the Redis server and Postgres services are up, run overmind start with the following "Procfile":

assets:  ./bin/webpack-dev-server
web:     bundle exec rails server
redis-server:   redis-server redis/redis.conf
worker:  bundle exec sidekiq -C config/sidekiq.yml

Containerized prod mode [branch "docker-prod"]

We will compose a production containerized mode, and launch a Sidekiq service (linked to a Redis server), a Postgres database service and the Rails app served with the app-server Puma, the latter serves the compiled static files as per our configuration.

In real life, the Redis service will most probably be a managed solution, a remote service, so you may not need to create a Redis service. Here, we maintain a custom Redis database. You can test a managed service by commenting off the Redis service and dependencies in the "docker-compose.yml" file and pass the remote URL from the managed service by specifying ENV['REDIS_URL'] in the code for both Sidekiq and Redis (see note at the end). We used for example redislabs.
The same remark applies to the Postgres service. We will most probably use a managed service rather than running your own server on bare metal, so you can comment off this service as well and it's dependencies (such as depends_on: keys) and use the remote url. For example, we can use ENV['ELEPHANT_URL'] when pointing to the managed service ElephantSQL.

To run these four images, we will need to build and run them within an internal network. This is done with the help of "docker-compose".

  • Rails image is based on the Dockerfile and launched with bundle exec rails server with an open port,
  • the background job processor Sidekiq uses the same image as Rails but launches with bundle exec sidekiq with a config file to link to the Redis session used.
  • the (optional) custom Redis database uses the official Redis image launched with redis-server and uses a config file (password, persistence)...).
  • the Postgres database uses the official Postgres image with an initialization script ("init-user.sql").

Note that this is not really made for production. It is a validation step before deploying with Kubernetes. With managed services, this app deployed on Kubernetse would only use one image, for the Rails app and Sidekiq.

Initializing, Volumes and bindings

In the compose file, we use two mount binds:

  • an SQL initializer for Postgres
  • a Redis config file

The Postgres initializer "init-user.sql" will create a user with password when the data directory is empty. This is done by pushing an file in the "/docker-entrypoint-initdb.d" folder (see "Initialization scripts"). It is run whenever the "data" folder is empty and the script is made idempotent (see code at the end). These credentials will be used as environment variables to configure Postgres in Rails and used in the "docker-compose.yml" file.

The two other named volumes are relative to the main data of the databases Postgres (PG_DATA=/var/lib/postgresql/data) and Redis (/data).

The Redis database can or not be used by Sidekiq for managing its queue. We can run two distinct Redis sessions (see section on remote services at the end).

  • the "sidekiq" service is initialized in the file "/config/initializers/sidekiq.rb" where we provide the link to a Redis session via ENV['SIDEKIQ_REDIS'].

  • the "app" service initializes the Redis in-memory database in a ad-hoc "/config.redis.yml" file where we pass ENV['REDIS_URL'].
    It is initialized with a "redis.conf" file that holds the password requirepass secretpwd (circa line 500).

Since "sidekiq" and "app" are built with the same image, the app code holds the environment variables REDIS_URL and SIDEKIQ_REDIS in the ".env" file. This ".env" file is passed to the "app", "sidekiq" and "redis" services.

  • migration. Once the Postgres database has been initialized with a "user|password", the "app" service has an entrypoint file "manage-db.sh" that performs the idempotent command rake db:prepare to create and migrate the database. Again, we need to pass the same credentials with the environment variable POSTGRES_PASSWORD=<password> in the docker-compose.yml file and save them in the ".env" file in the code (see note at the end).

The "docker-compose" production file 

version: "3"

x-app: &common
  env_file: .env
  build:
    context: .
    dockerfile: alpine.prod.Dockerfile
    args:
      - RUBY_VERSION=3.0.1-alpine
      - BUNDLER_VERSION=2.2.21
      - NODE_ENV=production
      - RAILS_ENV=production

services:
  pg:
    image: postgres:13.3-alpine
    ports:
      - 5432
    environment:
      - POSTGRES_PASSWORD=dockerpassword
    volumes:
      - pg_data:/var/lib/postgresql/data
      - ./pg/init.sql:/docker-entrypoint-initdb.d/init.sql:ro
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U postgres"]
      interval: 5s
      timeout: 5s

  redisdb:
    build:
      context: ./redis
    ports:
      - 6379
    env_file: .env
    command: ["redis-server", "/usr/local/etc/redis.conf"]
    volumes:
      - redis_data:/data
      - ./redis/config:/usr/local/etc/redis:ro
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 30s
      timeout: 10s
      retries: 3

  sidekiq:
    <<: *common
    entrypoint: ["./app-pid.sh"]
    command: ["bundle", "exec", "sidekiq", "-C", "config/sidekiq.yml"]
    depends_on:
      redisdb:
        condition: service_health


  app:
    <<: *common
    depends_on:
      pg:
        condition: service_healthy
      redisdb:
        condition: service_healthy
    ports:
      - 4000:3000
    entrypoint:  [./manage-db.sh]
    command: ["bundle", "exec", "rails", "server", "-b", "0.0.0.0"]
    tmpfs:
      - /tmp

volumes:
  redis_data: {}
  pg_data: {}

We run:

docker-compose up --build

If we don't use the "manage-db.sh" entrypoint, we would need to run:

docker-compose run --rm app bundle exec rails db:prepare

to setup the database and get the app up and running.

We added the package netcat in the Dockerfile since we use nc for health testing. This is for testing purposes and should be removed when using a managed service.

The "development" "docker-compose.yml" file

For the development mode, we define five processes:

  • the Rails app in dev mode for hot-reload,
  • the Sidekiq background process
  • the Webpack static assets manager run with webpacker-dev-server for hot-reload,
  • the Redis database adapter,
  • the Postgres database adapter

We use mount bindings for hot-reload and faster loading.

version: "3"

x-app: &common
  env_file: .env
  build:
    context: .
    args:
      - RUBY_VERSION=3.0.1-alpine
      - BUNDLER_VERSION=2.2.21
      #- NODE_VERSION=14 <- for "slim-buster" based image
      - NODE_ENV=development
      - RAILS_ENV=development

services:
  pg:
    image: postgres:13.3-alpine
    ports:
      - 5432
    environment:
      - POSTGRES_PASSWORD=cyberdyne
      - PG_DATA=/var/lib/postgresql/data
    volumes:
      - pg_data:/var/lib/postgresql/data
      - ./pg/init.sql:/docker-entrypoint-initdb.d/init.sql
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U postgres"]
      interval: 5s
      timeout: 5s
      retries: 5

  redisdb:
    build:
      context: ./redis
    ports:
      - 6379
    env_file: .env
    command: ["redis-server", "/usr/local/etc/redis.conf"]
    volumes:
      - redis_data:/data
      - ./redis/config:/usr/local/etc/redis
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 30s
      timeout: 10s
      retries: 3

  webpacker:
    <<: *common
    command: ["./bin/webpack-dev-server"]
    ports:
      - "3035:3035"
    volumes:
      - .:/app:cached
      - packs:/app/public/packs
    environment:
      - WEBPACKER_DEV_SERVER_HOST=0.0.0.0

  sidekiq:
    <<: *common
    entrypoint: ["./app-pid.sh"]
    command: ["bundle", "exec", "sidekiq", "-C", "config/sidekiq.yml"]
    depends_on:
      redisdb:
        condition: service_healthy

  app:
    <<: *common
    depends_on:
      pg:
        condition: service_healthy
      redisdb:
        condition: service_healthy
    environment:
      YARN_CACHE_FOLDER: /app/node_modules/.yarn-cache
      WEBPACKER_DEV_SERVER_HOST: webpacker
    ports:
      - 4000:3000
    entrypoint: ["./manage-db.sh"]
    command: ["bundle", "exec", "rails", "server", "-b", "0.0.0.0"]
    volumes:
      - bundle_cache:/usr/local/bundle
      - node_modules:/app/node_modules
      - .:/app:cached
      - packs:/app/public/packs
      - rails_cache:/app/tmp/cache
    tmpfs:
      - /tmp

volumes:
  redis_data: {}
  pg_data: {}
  node_modules:
  packs:
  bundle_cache:
  rails_cache:

Misc.

The "slim-buster" Dockerfile version 

ARG RUBY_VERSION
FROM ruby:${RUBY_VERSION} AS builder

ARG BUNDLER_VERSION \
   NODE_VERSION \
   NODE_ENV \
   RAILS_ENV

ENV RAILS_ENV=${RAILS_ENV} \
   NODE_ENV=${NODE_ENV} \
   BUNDLER_VERSION=${BUNDLER_VERSION}

RUN apt-get update \
   # && DEBIAN_FRONTEND=noninteractive 
   && apt-get install -y --no-install-recommends \
   # for gems to be compiled
   build-essential \
   # to get desired node version
   curl \
   && apt-get clean \
   && rm -rf /var/cache/apt/archives/* \
   && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* \
   && truncate -s 0 /var/log/*log

RUN curl -sL https://deb.nodesource.com/setup_${NODE_VERSION}.x | bash - \
   && curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | apt-key add - \
   && echo 'deb https://dl.yarnpkg.com/debian/ stable main' > /etc/apt/sources.list.d/yarn.list

RUN apt-get  update  \
   # && DEBIAN_FRONTEND=noninteractive 
   && apt-get install -y --no-install-recommends \
   # comm with PG with gem 'pg'
   libpq-dev \ 
   # compile assets
   nodejs \
   yarn \
   && apt-get clean \
   && rm -rf /var/cache/apt/archives/* \
   && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* \
   && truncate -s 0 /var/log/*log

WORKDIR /app

COPY Gemfile Gemfile.lock package.json yarn.lock ./

ENV LANG=C.UTF-8 \
   BUNDLE_JOBS=4 \
   BUNDLE_RETRY=3 \
   BUNDLE_PATH='vendor/bundle'

RUN gem install bundler:${BUNDLER_VERSION} --no-document \
   && bundle config set --without 'development test' \
   && bundle install --quiet \
   && rm -rf /usr/local/bundle/cache/*.gem \
   && find /usr/local/bundle/gems/ -name "*.c" -delete \
   && find /usr/local/bundle/gems/ -name "*.o" -delete

RUN yarn --check-files --silent

COPY . ./

RUN bundle exec rake assets:precompile
# && rm -rf node_modules tmp/cache app/assets vendor/assets lib/assets spec

###########################################
ARG RUBY_VERSION
FROM ruby:${RUBY_VERSION}

ARG NODE_ENV
ARG RAILS_ENV

RUN apt-get  update  \
   && apt-get install -y --no-install-recommends \
   # detect when services inside containers are up and running
   netcat-openbsd \
   # communicate with PG with gem 'pg'
   libpq-dev \
   && apt-get clean \
   && rm -rf /var/cache/apt/archives/* \
   && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* \
   && truncate -s 0 /var/log/*log


#<- if didn't used the flag BUNBLER_PATH='vendor/bundle', we would copy the host bundle folder
# COPY --from=builder /usr/local/bundle/ /usr/local/bundle/

RUN adduser --disabled-password app-user
USER app-user

COPY --from=builder  --chown=app-user /app /app

ENTRYPOINT ["./app-pid.sh"]

ENV RAILS_ENV=$RAILS_ENV\
   NODE_ENV=$NODE_ENV \
   RAILS_LOG_TO_STDOUT=true \
   RAILS_SERVE_STATIC_FILES=true \
   BUNDLE_PATH='vendor/bundle'

WORKDIR /app
RUN rm -rf node_modules tmp/cache  lib/assets

EXPOSE 3000

Remote services

If you want to rely on a remote Postgres or Redis service, you need to configure your account and pass the provided URL to the app.

For example, with a remote (free) service ElephantSQL, pass the supplied ENV[ELEPHANT_URL] into "/config/database.yml" and set the supplied POSTGRES_PASSWORD=xxxx variable in the "docker-compose.yml" file.

We can also use:

  • a local containerized Redis server for Sidekiq's queue. Set SIDEKIQ_REDIS=redis://user:password@redisdb:6379 in the file "/config/sidekiq.rb"

  • a remote Redis database for the app. We used a (free) service supplied by Redis Labs. Set url: <%= ENV.fetch('REDIS_URL','') %> with the supplied URL from Redislabs in the file "/config/redis.yml". This can replace the Redis service.

".env" file

An example of the environment variables used:

#.env
export POSTGRES_URL=postgresql://docker:dockerpassword@pg:5432
export ELEPHANT_URL=postgres://ortkcbqt:fhSBQrF3Dzl9WWA1FfRIjQmU7u3pBtTd@batyr.db.elephantsql.com/ortkcbqt

export SIDEKIQ_REDIS=redis://user:secretpwd@redisdb:6379
export REDIS_URL=redis://user:tq4hBlYvIvq0uU7hYMOYS6ErQKsSA2N8@redis-13424.c258.us-east-1-4.ec2.cloud.redislabs.com:13424

export RAILS_MASTER_KEY=cce3c51968fc41dd85b3d8b5d54f43eb

export RAILS_SERVE_STATIC_FILES=true
export RAILS_LOG_TO_STDOUT=true

Postgres USER initializer

This code is run whenever the "data" directory is empty to declare a with (the "<" & ">" signs are just here to emphasis).

#init.sql
DO $$
BEGIN
  CREATE ROLE <docker> WITH SUPERUSER CREATEDB LOGIN PASSWORD <'dockerpassword'>;
  EXCEPTION WHEN DUPLICATE_OBJECT THEN
  RAISE NOTICE 'not creating role my_role -- it already exists';
END
$$;

The environment variable POSTGRES_URL=postgresql://<user>:<password>@pg:5423 passed to the "database.yml" in the code must match the "|" used in the "init-user.sql" file and we must pass POSTGRES_PASSWORD=<pasword> in the "docker-compose.yml" file.
In you compose with non matching credentials, then you need to run docker volume prune and rebuild.

Entrypoints

The rake command is idempotent.

#manage-db.sh <-- "app" service

#!/bin/sh
set -e
if [ -f tmp/pids/server.pid ]; then
  rm tmp/pids/server.pid
fi
echo "Waiting for Postgres to start..."
while ! nc -z pg 5432; do sleep 0.2; done
echo "Postgres is up"

bundle exec rake db:prepare
exec "$@"

This code is the entrypoint of the "sidekiq" service. It "cleans" the "pid" file.

#app-pid.sh <-- "sidekiq" service

#!/bin/sh
set -e
if [ -f tmp/pids/server.pid ]; then
  rm tmp/pids/server.pid
fi
exec "$@"

Redis database initializer

We create a file "/config/initializers/redis.rb" so that Rails will load it on startup and instantiate a Redis session.

#config/initializers/redis.rb
REDIS = Redis.new(Rails.application.config_for(:redis))

where config_for will parse and fetch the "/config/redis.yml" config so we can use for example REDIS.set("key", 10) in the app.

We can test the protection of the Redis database with the command below once the project is up and running:details here:

docker-compose run --rm redisdb redis-cli -h redisdb -p 6379

The output should be:

redisdb:6379> get compteur
(error) NOAUTH Authentication required.
redisdb:6379> auth secretpwd
OK
redisdb:6379> get compteur
"3"

The Redis persistence is parametered by configuring the "redis.conf" file:

  • RDB mode periodic: dbfilename "dump.rdb"(line 327)
  • AOF mode: every seconde with appendonly yes (line 699) and appendfsync everysec (line 729) and appendfilename appendonly.aof(line 703)

Hope this help!


This content originally appeared on DEV Community and was authored by NDREAN


Print Share Comment Cite Upload Translate Updates
APA

NDREAN | Sciencx (2021-07-02T18:02:22+00:00) Make slim Rails images. Retrieved from https://www.scien.cx/2021/07/02/make-slim-rails-images/

MLA
" » Make slim Rails images." NDREAN | Sciencx - Friday July 2, 2021, https://www.scien.cx/2021/07/02/make-slim-rails-images/
HARVARD
NDREAN | Sciencx Friday July 2, 2021 » Make slim Rails images., viewed ,<https://www.scien.cx/2021/07/02/make-slim-rails-images/>
VANCOUVER
NDREAN | Sciencx - » Make slim Rails images. [Internet]. [Accessed ]. Available from: https://www.scien.cx/2021/07/02/make-slim-rails-images/
CHICAGO
" » Make slim Rails images." NDREAN | Sciencx - Accessed . https://www.scien.cx/2021/07/02/make-slim-rails-images/
IEEE
" » Make slim Rails images." NDREAN | Sciencx [Online]. Available: https://www.scien.cx/2021/07/02/make-slim-rails-images/. [Accessed: ]
rf:citation
» Make slim Rails images | NDREAN | Sciencx | https://www.scien.cx/2021/07/02/make-slim-rails-images/ |

Please log in to upload a file.



There are no updates yet.
Click the Upload button above to add an update.

You must be logged in to translate posts. Please log in or register.

Related Posts