Writing a Dockerfile for catscii

Mar 5, 2023 10 minute read rust · docker

This article is part of the Building a Rust service with Nix series.

Installing Docker on Ubuntu
A basic Dockerfile to build our service
Installing fewer packages
Reclaiming Docker disk space
Building the service for real
Caching some directories
More missing dependencies
Quick rant about the "Ubuntu CDN"
Running our application with Docker
Ctrl-C doesn't exit my docker container
Implementing graceful shutdown

Now that our service is production-ready, it's time to deploy it somewhere.

There's a lot of ways to approach this: what we are going to do, though, is build a docker image. Or, I should say, an OCI image.

This is still a series about Nix, but again: because the best way to see the benefits of Nix is to do it without Nix first, we'll use only Docker's tooling to build the image.

Installing Docker on Ubuntu

Docker has its own docs about that. Many times I've followed all the instructions, only to realize near the very end that they actually have a convenience script.

Cool bear's hot tip

There's also docker packages in the Ubuntu repositories, but it tends to lag behind official releases by quite a bit, so we won't bother with it.

From VS Code's integrated terminal, we can run the convenience script:

Shell session

amos@miles:~/catscii$ cd ~
direnv: unloading
amos@miles:~$ curl -fsSL https://get.docker.com -o get-docker.sh
amos@miles:~$ sudo sh get-docker.sh
[sudo] password for amos: 
# Executing docker install script, commit: 4f282167c425347a931ccfd95cc91fab041d414f
+ sh -c apt-get update -qq >/dev/null
+ sh -c DEBIAN_FRONTEND=noninteractive apt-get install -y -qq apt-transport-https ca-certificates curl >/dev/null
+ sh -c mkdir -p /etc/apt/keyrings && chmod -R 0755 /etc/apt/keyrings
+ sh -c curl -fsSL "https://download.docker.com/linux/ubuntu/gpg" | gpg --dearmor --yes -o /etc/apt/keyrings/docker.gpg
+ sh -c chmod a+r /etc/apt/keyrings/docker.gpg
+ sh -c echo "deb [arch=amd64 signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu jammy stable" > /etc/apt/sources.list.d/docker.list
+ sh -c apt-get update -qq >/dev/null
+ sh -c DEBIAN_FRONTEND=noninteractive apt-get install -y -qq --no-install-recommends docker-ce docker-ce-cli containerd.io docker-compose-plugin docker-scan-plugin >/dev/null
+ version_gte 20.10
+ [ -z  ]
+ return 0
+ sh -c DEBIAN_FRONTEND=noninteractive apt-get install -y -qq docker-ce-rootless-extras >/dev/null
+ sh -c docker version
Client: Docker Engine - Community
 Version:           23.0.6
 API version:       1.41
 Go version:        go1.18.7
 Git commit:        baeda1f
 Built:             Tue Oct 25 18:01:58 2022
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          23.0.6
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.18.7
  Git commit:       3056208
  Built:            Tue Oct 25 17:59:49 2022
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.10
  GitCommit:        770bd0108c32f3fb5c73ae1264f7e503fe7b2661
 runc:
  Version:          1.1.4
  GitCommit:        v1.1.4-0-g5fd4c4d
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

================================================================================

To run Docker as a non-privileged user, consider setting up the
Docker daemon in rootless mode for your user:

    dockerd-rootless-setuptool.sh install

Visit https://docs.docker.com/go/rootless/ to learn about rootless mode.


To run the Docker daemon as a fully privileged service, but granting non-root
users access, refer to https://docs.docker.com/go/daemon-access/

WARNING: Access to the remote API on a privileged Docker daemon is equivalent
         to root access on the host. Refer to the 'Docker daemon attack surface'
         documentation for details: https://docs.docker.com/go/attack-surface/

================================================================================

Once that's done, we can follow the Linux post-install steps:

Shell session

amos@miles:~$ sudo usermod -aG docker $USER

Cool bear's hot tip

As mentioned in the Docker docs, The docker group grants root-level privileges to the user. For details on how this impacts security in your system, see Docker Daemon Attack Surface.

After that, we can reboot the virtual machine, or, until we do, we can make sure we run newgrp docker in any terminal we want to use docker in.

That should allow us to run docker images.

Shell session

amos@miles:~$ newgrp docker
amos@miles:~$ docker run hello-world
Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
2db29710123e: Pull complete 
Digest: sha256:faa03e786c97f07ef34423fccceeec2398ec8a5759259f94d99078f264e9d7af
Status: Downloaded newer image for hello-world:latest

Hello from Docker!
This message shows that your installation appears to be working correctly.

To generate this message, Docker took the following steps:
 1. The Docker client contacted the Docker daemon.
 2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
    (amd64)
 3. The Docker daemon created a new container from that image which runs the
    executable that produces the output you are currently reading.
 4. The Docker daemon streamed that output to the Docker client, which sent it
    to your terminal.

To try something more ambitious, you can run an Ubuntu container with:
 $ docker run -it ubuntu bash

Share images, automate workflows, and more with a free Docker ID:
 https://hub.docker.com/

For more examples and ideas, visit:
 https://docs.docker.com/get-started/

All good, time to write a Dockerfile!

A basic `Dockerfile` to build our service

We've developed our service on Ubuntu, and we can use it as a base for our image, too.

I forget, what version of Ubuntu are we using again?

Ah well, we can use lsb_release to find out:

Shell session

amos@miles:~/catscii$ lsb_release --all
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 22.04.1 LTS
Release:        22.04
Codename:       jammy

But if we didn't have it (since it doesn't ship in all distributions by default), we could just read the contents of /etc/os-release:

Shell session

amos@miles:~/catscii$ cat /etc/os-release 
PRETTY_NAME="Ubuntu 22.04.1 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04.1 LTS (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy

So, let's create a Dockerfile in the root of our repository, and... try to install Rust in there, for a start:

Dockerfile

FROM ubuntu:22.04

# Install rustup
RUN set -eux; \
		curl --location --fail \
			"https://static.rust-lang.org/rustup/dist/x86_64-unknown-linux-gnu/rustup-init" \
			--output rustup-init; \
		chmod +x rustup-init; \
		./rustup-init -y --no-modify-path --default-toolchain stable; \
		rm rustup-init;

# Add rustup to path, check that it works
ENV PATH=${PATH}:/root/.cargo/bin
RUN set -eux; \
		rustup --version;

Note, if you encounter an error related to rustup not being able to choose a version of cargo during the cargo build --release step, you might need to ensure that the rustup cache is available during the rustup installation. Modify the rustup installation command in your Dockerfile as follows:

Dockerfile

RUN --mount=type=cache,target=/root/.rustup \
    set -eux; \
    curl --location --fail \
      "https://static.rust-lang.org/rustup/dist/x86_64-unknown-linux-gnu/rustup-init" \
      --output rustup-init; \
    chmod +x rustup-init; \
    ./rustup-init -y --no-modify-path --default-toolchain stable; \
    rm rustup-init;

This ensures that the rustup cache is mounted, potentially resolving the issue.

Cool bear's hot tip

set -eux allows running multiple commands in a single RUN statement (which then counts as a single layer), while printing all commands (-x), erroring if a variable we use isn't bound (-u), and failing if one of the commands fail (-e).

And try to build it:

Shell session

amos@miles:~/catscii$ docker build --tag catscii .
^Cnding build context to Docker daemon  187.7MB

No, wait, abort! (Seriously, hit Ctrl-C). It's transferring hundreds of megabytes to the Docker daemon, something's wrong.

But... we don't have hundreds of megabytes of source code?

No we don't. But we have a few gigabytes of artifacts in target/:

Shell session

amos@miles:~/catscii$ du --summarize --human-readable target/
3.6G    target/

Oh wow. Debug info is heavy! target/debug/ is 2.6G just by itself.

Yeah. Some of it is just due to iterating though — doing cargo clean && cargo build brings it down to "just" 1.8G, but yes, debug info is heavy.

Anyway we don't want that target directory to be part of Docker's build context, so, much like we ignore it in .gitignore, we can ignore it in .dockerignore:

# in `.dockerignore`
/target

Shell session

amos@miles:~/catscii$ docker build --tag catscii .
Sending build context to Docker daemon  447.5kB
Step 1/4 : FROM ubuntu:20.04
20.04: Pulling from library/ubuntu
eaead16dc43b: Pull complete 
Digest: sha256:450e066588f42ebe1551f3b1a535034b6aa46cd936fe7f2c6b0d72997ec61dbd
Status: Downloaded newer image for ubuntu:20.04
 ---> 680e5dfb52c7
Step 2/4 : RUN set -eux;                curl --location --fail                  "https://static.rust-lang.org/rustup/dist/x86_64-unknown-linux-gnu/rustup-init"                         --output rustup-init;          chmod +x rustup-init;           ./rustup-init -y --no-modify-path --default-toolchain stable;           rm rustup-init;
 ---> Running in ce4072a4edf5
+ curl --location --fail https://static.rust-lang.org/rustup/dist/x86_64-unknown-linux-gnu/rustup-init --output rustup-init
/bin/sh: 1: curl: not found
The command '/bin/sh -c set -eux;               curl --location --fail                  "https://static.rust-lang.org/rustup/dist/x86_64-unknown-linux-gnu/rustup-init"                         --output rustup-init;          chmod +x rustup-init;           ./rustup-init -y --no-modify-path --default-toolchain stable;           rm rustup-init;' returned a non-zero code: 127

Ah but of course. The base ubuntu:20.04 image doesn't have curl.

Well, no matter, let's install it:

Dockerfile

FROM ubuntu:20.04

# Install curl
RUN set -eux; \
		apt install -y curl;

# Install rustup
# (cut: same as before)

Shell session

$ docker build --tag catscii .
Sending build context to Docker daemon  453.1kB
Step 1/5 : FROM ubuntu:20.04
 ---> 680e5dfb52c7
Step 2/5 : RUN set -eux;                apt install -y curl;
 ---> Running in 7291a0f7bfa6
+ apt install -y curl

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

Reading package lists...
Building dependency tree...
Reading state information...
E: Unable to locate package curl
The command '/bin/sh -c set -eux;               apt install -y curl;' returned a non-zero code: 100

Ah yes. The ubuntu docker image doesn't actually have APT repository information: that's been culled to reduce the image size. So we have to run "apt update".

Cool bear's hot tip

"apt" is a different (more modern) command than "apt-get", and the former warns to "use with caution in scripts", since its output isn't stable. But we're not parsing the output of either command, so it's fine.

Let's amend our Dockerfile to:

Dockerfile

# Install curl
RUN set -eux; \
		apt update; \
		apt install -y curl;

This time, we get this:

Shell session

$ docker build --tag catscii .
Sending build context to Docker daemon  453.1kB
Step 1/5 : FROM ubuntu:20.04
 ---> d5447fc01ae6
Step 2/5 : RUN set -eux;                apt update;             apt install -y curl;
 ---> Running in 13081cf88d73
+ apt update

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

Get:1 http://archive.ubuntu.com/ubuntu focal InRelease [265 kB]
(cut)
Get:18 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [2358 kB]
Fetched 24.7 MB in 10s (2567 kB/s)
Reading package lists...
Building dependency tree...
Reading state information...
All packages are up to date.
+ apt install -y curl

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

Reading package lists...
Building dependency tree...
Reading state information...
The following additional packages will be installed:
  ca-certificates krb5-locales libasn1-8-heimdal libbrotli1 libcurl4
  libgssapi-krb5-2 libgssapi3-heimdal libhcrypto4-heimdal libheimbase1-heimdal
  libheimntlm0-heimdal libhx509-5-heimdal libk5crypto3 libkeyutils1
  libkrb5-26-heimdal libkrb5-3 libkrb5support0 libldap-2.4-2 libldap-common
  libnghttp2-14 libpsl5 libroken18-heimdal librtmp1 libsasl2-2
  libsasl2-modules libsasl2-modules-db libsqlite3-0 libssh-4 libssl1.1
  libwind0-heimdal openssl publicsuffix
Suggested packages:
  krb5-doc krb5-user libsasl2-modules-gssapi-mit
  | libsasl2-modules-gssapi-heimdal libsasl2-modules-ldap libsasl2-modules-otp
  libsasl2-modules-sql
The following NEW packages will be installed:
  ca-certificates curl krb5-locales libasn1-8-heimdal libbrotli1 libcurl4
  libgssapi-krb5-2 libgssapi3-heimdal libhcrypto4-heimdal libheimbase1-heimdal
  libheimntlm0-heimdal libhx509-5-heimdal libk5crypto3 libkeyutils1
  libkrb5-26-heimdal libkrb5-3 libkrb5support0 libldap-2.4-2 libldap-common
  libnghttp2-14 libpsl5 libroken18-heimdal librtmp1 libsasl2-2
  libsasl2-modules libsasl2-modules-db libsqlite3-0 libssh-4 libssl1.1
  libwind0-heimdal openssl publicsuffix
0 upgraded, 32 newly installed, 0 to remove and 0 not upgraded.
Need to get 5443 kB of archives.
After this operation, 16.7 MB of additional disk space will be used.
Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl1.1 amd64 1.1.1f-1ubuntu2.16 [1321 kB]
(cut)
Get:32 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libsasl2-modules amd64 2.1.27+dfsg-2ubuntu0.1 [48.8 kB]
debconf: delaying package configuration, since apt-utils is not installed
Fetched 5443 kB in 3s (1879 kB/s)
Selecting previously unselected package libssl1.1:amd64.
(Reading database ... 4126 files and directories currently installed.)
Preparing to unpack .../00-libssl1.1_1.1.1f-1ubuntu2.16_amd64.deb ...
Unpacking libssl1.1:amd64 (1.1.1f-1ubuntu2.16) ...
Selecting previously unselected package openssl.
(cut)
Setting up curl (7.68.0-1ubuntu2.14) ...
Processing triggers for libc-bin (2.31-0ubuntu9.9) ...
Processing triggers for ca-certificates (20211016ubuntu0.20.04.1) ...
Updating certificates in /etc/ssl/certs...
0 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d...
done.
Removing intermediate container 13081cf88d73
 ---> 01eb9703cd62
Step 3/5 : RUN set -eux;                curl --location --fail                  "https://static.rust-lang.org/rustup/dist/x86_64-unknown-linux-gnu/rustup-init"       --output rustup-init;            chmod +x rustup-init;           ./rustup-init -y --no-modify-path --default-toolchain stable;           rm rustup-init;
 ---> Running in 47b200bdfc62
+ curl --location --fail https://static.rust-lang.org/rustup/dist/x86_64-unknown-linux-gnu/rustup-init --output rustup-init
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 15.0M  100 15.0M    0     0  13.8M      0  0:00:01  0:00:01 --:--:-- 13.8M
+ chmod +x rustup-init
+ ./rustup-init -y --no-modify-path --default-toolchain stable
info: profile set to 'default'
info: default host triple is x86_64-unknown-linux-gnu
info: syncing channel updates for 'stable-x86_64-unknown-linux-gnu'
info: latest update on 2022-12-15, rust version 1.66.0 (69f9c33d7 2022-12-12)
info: downloading component 'cargo'
info: downloading component 'clippy'
info: downloading component 'rust-docs'
info: downloading component 'rust-std'
info: downloading component 'rustc'
info: downloading component 'rustfmt'
info: installing component 'cargo'
info: installing component 'clippy'
info: installing component 'rust-docs'
info: installing component 'rust-std'
info: installing component 'rustc'
info: installing component 'rustfmt'
info: default toolchain set to 'stable-x86_64-unknown-linux-gnu'

  stable-x86_64-unknown-linux-gnu installed - rustc 1.66.0 (69f9c33d7 2022-12-12)


Rust is installed now. Great!

To get started you need Cargo's bin directory ($HOME/.cargo/bin) in your PATH
environment variable. This has not been done automatically.

To configure your current shell, run:
source "$HOME/.cargo/env"
+ rm rustup-init
Removing intermediate container 47b200bdfc62
 ---> 7609ba153931
Step 4/5 : ENV PATH=${PATH}:/root/.cargo/bin
 ---> Running in 9cadc26748ba
Removing intermediate container 9cadc26748ba
 ---> 09f5615bbe83
Step 5/5 : RUN set -eux;                rustup --version;
 ---> Running in 1ab7d2570fc1
+ rustup --version
info: This is the version for the rustup toolchain manager, not the rustc compiler.
rustup 1.25.1 (bb60b1e89 2022-07-12)
info: The currently active `rustc` version is `rustc 1.66.0 (69f9c33d7 2022-12-12)`
Removing intermediate container 1ab7d2570fc1
 ---> 90d38ea8bd14
Successfully built 90d38ea8bd14
Successfully tagged catscii:latest

So now, in that docker image, we have rustc!

Shell session

$ docker run --rm catscii rustc --version
rustc 1.66.0 (69f9c33d7 2022-12-12)

Which is good and bad. It's good, because we need rustc to build our service, but it's bad, because we don't need rustc at runtime. We'll get to that later.

First let's get the service building at all:

Dockerfile

# (After adding rustup to path)

# Copy sources and build them
WORKDIR /app
COPY src src
COPY Cargo.toml Cargo.lock ./
RUN set -eux; \
		cargo build --release;

Cool bear's hot tip

The ADD command can also copy stuff from the "build context" to the "builder", but Docker documentation recommends against that.

The COPY command has somewhat surprising behavior when passed directories. The following:

Dockerfile

COPY src Cargo.toml Cargo.lock ./

...doesn't create a src/ directory in the builder. It copies the contents of the src folder there. Hence the two separate COPY invocations.

Shell session

$ docker build --tag catscii .
(cut)
error: linker `cc` not found
  |
  = note: No such file or directory (os error 2)

So, again there's two problems here. The long-term problem is that when compiling, cargo needs to fetch the crates.io index. And because of the way the Dockerfile is set up, it'll have to do so from scratch every time.

Also, at the time of this writing, sparse registries are not the default (even though they work pretty well already), so in concrete terms, that means cloning a large Git repository every time.

But we'll get to that later: the main problem is that we're getting an error: it says it can't find the cc linker.

Fine then, let's try installing gcc at the same time we install curl:

Dockerfile

# Install required dependencies
RUN set -eux; \
		apt update; \
		apt install -y curl gcc;

# (cut: everything else)

Because the layer we're changing is at the very beginning of our Dockerfile, it's going to do everything again, including installing rustup, cloning the crates.io registry, etc.

Hope you have a fast internet connection!

Cool bear's hot tip

The fact that we're playing "Docker golf" right now (because there's so many things to get right) explains why there's a bit of a copy-paste culture around Dockerfiles.

If you have a Dockerfile that works, it's tempting to copy it for other projects, and the end result is large files, with a lot of parts that nobody's quite sure why.

Installing fewer packages

I've had time to check all my messages and it's still downloading gcc and friends. I guess I'll cancel the build and show you a little APT tip: APT has "recommended" and "suggested" packages. By default, it installs the "recommended" packages, which in this case, is a lot of packages.

We can disable that behavior with --no-install-recommends:

Dockerfile

# Install required dependencies
RUN set -eux; \
		apt update; \
		apt install -y --no-install-recommends \
			curl gcc \
			;

Cool bear's hot tip

This tip is Ubuntu-specific. Other distributions have other package managers that work differently. But Ubuntu is a typical enough server-side target that it's not that surprising that this series focuses on it.

Reclaiming Docker disk space

Note that at this point, because we've built the image a bunch of times, Docker is using a bunch of disk space for nothing: since we've changed the first layer, we're not going to be re-using anything in cache.

Shell session

$ docker system df
TYPE            TOTAL     ACTIVE    SIZE      RECLAIMABLE
Images          4         3         1.286GB   1.286GB (99%)
Containers      5         0         251.7MB   251.7MB (100%)
Local Volumes   0         0         0B        0B
Build Cache     0         0         0B        0B

We can reclaim that space with docker system prune:

Shell session

$ docker system prune
(cut)

deleted: sha256:492b6da4cea745e019df452f2c305a5b0569dc2c1451966171db9aa702f7418c

Total reclaimed space: 251.8MB

And even more space with --all, or -a for short (which deletes all unused images, not just dangling ones):

Shell session

$ docker system prune -a
(cut)

Total reclaimed space: 1.285GB

Building the service for real

Now, a lot fewer packages are installed, but it fails again with:

Shell session

$ docker build --tag catscii .
(cut)
 ---> Running in c6070d98b7e2
+ curl --location --fail https://static.rust-lang.org/rustup/dist/x86_64-unknown-linux-gnu/rustup-init --output rustup-init
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
curl: (77) error setting certificate verify locations:
  CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs

This occurs because we're using curl to download the rustup installer over HTTPS, and in order to verify the certificate, curl must consult a "root certificate bundle".

There's such a bundle in Ubuntu's ca-certificates package, but it's not a hard dependency of curl, merely a recommendation.

So... let's add it:

Dockerfile

# Install required dependencies
RUN set -eux; \
		apt update; \
		apt install -y --no-install-recommends \
			curl ca-certificates gcc \
			;

At this point, it gets as far as downloading the latest rust stable using rustup-init, but.. we should probably be able to cache that, right? It doesn't sound right to have to download that every time we change an earlier layer.

Caching some directories

BuildKit, the newer Docker builder backend, lets us mount some folders as "cache".

This is a "newer" Docker feature, so we'll need to opt into it within our Dockerfile with a load-bearing comment. Here's our full updated Dockerfile:

Dockerfile

# syntax = docker/dockerfile:1.4

# 👆 that there is the magic, load-bearing comment to opt into new features
# note using Docker 23.0.6 does not require the magic comment
FROM ubuntu:20.04

# Install required dependencies
RUN set -eux; \
		apt update; \
		apt install -y --no-install-recommends \
			curl ca-certificates gcc \
			;

# Install rustup
RUN set -eux; \
		curl --location --fail \
			"https://static.rust-lang.org/rustup/dist/x86_64-unknown-linux-gnu/rustup-init" \
			--output rustup-init; \
		chmod +x rustup-init; \
		./rustup-init -y --no-modify-path --default-toolchain stable; \
		rm rustup-init;

# Add rustup to path, check that it works
ENV PATH=${PATH}:/root/.cargo/bin
RUN set -eux; \
		rustup --version;

# Copy sources and build them
WORKDIR /app
COPY src src
COPY Cargo.toml Cargo.lock ./
#   👇 and here's some caches!
RUN --mount=type=cache,target=/root/.rustup \
    --mount=type=cache,target=/root/.cargo/registry \
    --mount=type=cache,target=/root/.cargo/git \
		--mount=type=cache,target=/app/target \
		set -eux; \
		cargo build --release;

Running this doesn't work:

Shell session

$ docker build --tag catscii .
Sending build context to Docker daemon  453.6kB
Step 1/9 : FROM ubuntu:20.04
 ---> d5447fc01ae6
Step 2/9 : RUN set -eux;                apt update;             apt install -y --no-install-recommends                  curl ca-certificates gcc                       ;
 ---> Using cache
 ---> d7d6c858dda7
Step 3/9 : RUN --mount=type=cache,target=/root/.cargo           --mount=type=cache,target=/root/.rustup           set -eux;             curl --location --fail         "https://static.rust-lang.org/rustup/dist/x86_64-unknown-linux-gnu/rustup-init"                  --output rustup-init;           chmod +x rustup-init;           ./rustup-init -y --no-modify-path --default-toolchain stable;           rm rustup-init;
the --mount option requires BuildKit. Refer to https://docs.docker.com/go/buildkit/ to learn how to build images with BuildKit enabled

Because, at least on the version of docker we installed into our Linux VM, BuildKit isn't enabled by default.

We can override this globally, or every time we run docker build:

Note, if you are running Docker 23.0.6 you can omit DOCKER_BUILDKIT=1

Shell session

$ DOCKER_BUILDKIT=1 docker build --tag catscii .

(continued in next section)

More missing dependencies

Shell session

$ DOCKER_BUILDKIT=1 docker build --tag catscii .
(cut)
ecstack" "-L" "/root/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-o" "/app/target/release/build/serde-387b017df630b297/build_script_build-387b017df630b297" "-Wl,--gc-sections" "-pie" "-Wl,-zrelro,-znow" "-nodefaultlibs"
#16 26.26   = note: /usr/bin/ld: cannot find Scrt1.o: No such file or directory
#16 26.26           /usr/bin/ld: cannot find crti.o: No such file or directory
#16 26.26           collect2: error: ld returned 1 exit status
#16 26.26           
#16 26.26 
#16 26.26 error: could not compile `serde` due to previous error
------
executor failed running [/bin/sh -c set -eux;           cargo build --release;]: exit code: 101

Well... that's also caused by --no-install-recommends. Did you know GCC has a runtime component? And that it's not a direct dependency of the gcc package on Ubuntu, but rather a "recommendation"? Yeah.

More golfing!

Dockerfile

# Install required dependencies
RUN set -eux; \
		apt update; \
		apt install -y --no-install-recommends \
    #                            👇 new!
			curl ca-certificates gcc libc6-dev \
			;

Our next error is...

Shell session

$ DOCKER_BUILDKIT=1 docker build --tag catscii .
(cut)
#16 2.196   run pkg_config fail: "Could not run `\"pkg-config\" \"--libs\" \"--cflags\" \"openssl\"`\nThe pkg-config command could not be found.\n\nMost likely, you need to install a pkg-config package for your OS.\nTry `apt install pkg-config`, or `yum install pkg-config`,\nor `pkg install pkg-config`, or `apk add pkgconfig` depending on your distribution.\n\nIf you've already installed it, ensure the pkg-config command is one of the\ndirectories in the PATH environment variable.\n\nIf you did not expect this build to link to a pre-installed system library,\nthen check documentation of the openssl-sys crate for an option to\nbuild the library from source, or disable features or dependencies\nthat require pkg-config."
#16 2.196 
#16 2.196   --- stderr
#16 2.196   thread 'main' panicked at '
#16 2.196 
#16 2.196   Could not find directory of OpenSSL installation, and this `-sys` crate cannot
#16 2.196   proceed without this knowledge. If OpenSSL is installed and this crate had
#16 2.196   trouble finding it,  you can set the `OPENSSL_DIR` environment variable for the
#16 2.196   compilation process.
#16 2.196 
#16 2.196   Make sure you also have the development packages of openssl installed.
#16 2.196   For example, `libssl-dev` on Ubuntu or `openssl-devel` on Fedora.
#16 2.196 
#16 2.196   If you're in a situation where you think the directory *should* be found
#16 2.196   automatically, please open a bug at https://github.com/sfackler/rust-openssl
#16 2.196   and include information about your system as well as this message.
#16 2.196 
#16 2.196   $HOST = x86_64-unknown-linux-gnu
#16 2.196   $TARGET = x86_64-unknown-linux-gnu
#16 2.196   openssl-sys = 0.9.78
#16 2.196 
#16 2.196 
#16 2.196   It looks like you're compiling on Linux and also targeting Linux. Currently this
#16 2.196   requires the `pkg-config` utility to find OpenSSL but unfortunately `pkg-config`
#16 2.196   could not be found. If you have OpenSSL installed you can likely fix this by
#16 2.196   installing `pkg-config`.
#16 2.196 
#16 2.196   ', /root/.cargo/registry/src/github.com-1ecc6299db9ec823/openssl-sys-0.9.78/build/find_normal.rs:191:5
#16 2.196   note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
#16 2.196 warning: build failed, waiting for other jobs to finish...

There's two problems here: we don't have pkg-config and we don't have the OpenSSL development package.

We could be golfing all day, let me fast forward to this point:

Dockerfile

# Install required dependencies
RUN set -eux; \
		apt update; \
		apt install -y --no-install-recommends \
			curl ca-certificates gcc libc6-dev pkg-config libssl-dev \
			;

Quick rant about the "Ubuntu CDN"

One of the reasons this is taking forever for me, by the way, is because it's grabbing everything from archive.ubuntu.com. It seems like they're doing DNS load balancing, but your luck may vary WILDLY from one run to the other.

Lucky run (landed somewhere in the UK):

Shell session

$ ping archive.ubuntu.com
PING archive.ubuntu.com (185.125.190.39) 56(84) bytes of data.
64 bytes from aerodent.canonical.com (185.125.190.39): icmp_seq=1 ttl=55 time=65.5 ms
64 bytes from aerodent.canonical.com (185.125.190.39): icmp_seq=2 ttl=55 time=20.3 ms
64 bytes from aerodent.canonical.com (185.125.190.39): icmp_seq=3 ttl=55 time=19.4 ms
64 bytes from aerodent.canonical.com (185.125.190.39): icmp_seq=4 ttl=55 time=26.2 ms
^C
--- archive.ubuntu.com ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3011ms
rtt min/avg/max/mdev = 19.400/32.853/65.472/19.014 ms

Unlucky run (landed in Boston Massachusetts, USA):

Shell session

$ ping archive.ubuntu.com
PING archive.ubuntu.com (91.189.91.38) 56(84) bytes of data.
64 bytes from banjo.canonical.com (91.189.91.38): icmp_seq=1 ttl=54 time=194 ms
64 bytes from banjo.canonical.com (91.189.91.38): icmp_seq=2 ttl=54 time=107 ms
64 bytes from banjo.canonical.com (91.189.91.38): icmp_seq=3 ttl=54 time=126 ms
^C
--- archive.ubuntu.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2009ms
rtt min/avg/max/mdev = 107.417/142.492/194.316/37.400 ms

These are all just Apache servers by the way:

Shell session

$ curl --head --silent archive.ubuntu.com | grep --ignore-case server
Server: Apache/2.4.29 (Ubuntu)

Well, at least they're dogfooding.

Running our application with Docker

Anyway, with the last changes, our application builds!

It, however, doesn't run:

Shell session

$ docker run --rm catscii /app/target/release/catscii
docker: Error response from daemon: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: exec: "/app/target/release/catscii": stat /app/target/release/catscii: no such file or directory: unknown.

That seems.. suspicious, let's start a shell inside the container to look around:

Shell session

$ docker run -it --rm catscii /bin/sh
# ls /app
Cargo.lock  Cargo.toml  src  target
# ls /app/target
#

Wasn't /target in a cache-mount?

Oh. Right 🙃 That explains why it doesn't actually end up in the final image.

It's also a bit annoying to have to specify the command we want to run, so we can just set a default in the Dockerfile:

Dockerfile

# (cut: magic syntax comment, installing dependencies, etc.)

# Copy sources and build them
WORKDIR /app
COPY src src
COPY Cargo.toml Cargo.lock ./
RUN --mount=type=cache,target=/root/.rustup \
    --mount=type=cache,target=/root/.cargo/registry \
    --mount=type=cache,target=/root/.cargo/git \
		--mount=type=cache,target=/app/target \
		set -eux; \
		cargo build --release; \
		cp target/release/catscii .

CMD ["/app/catscii"]

Shell session

$ docker build etc.
(cut)

$ docker run --rm catscii
thread 'main' panicked at '$SENTRY_DSN must be set: NotPresent', src/main.rs:37:37
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Oh hey! It runs! Let's pass it a few environment variables. They're already in our local environment, due to our .envrc / direnv setup, so we can just mention them by name:

Shell session

$ docker run --env SENTRY_DSN --env HONEYCOMB_API_KEY --rm catscii
{"timestamp":"2022-12-23T16:41:45.195202Z","level":"INFO","fields":{"message":"Creating honey client","log.target":"libhoney::client","log.module_path":"libhoney::client","log.file":"/root/.cargo/git/checkouts/libhoney-rust-3b5a30a6076b74c0/9871051/src/client.rs","log.line":78},"target":"libhoney::client"}
{"timestamp":"2022-12-23T16:41:45.195238Z","level":"INFO","fields":{"message":"transmission starting","log.target":"libhoney::transmission","log.module_path":"libhoney::transmission","log.file":"/root/.cargo/git/checkouts/libhoney-rust-3b5a30a6076b74c0/9871051/src/transmission.rs","log.line":124},"target":"libhoney::transmission"}
{"timestamp":"2022-12-23T16:41:45.199169Z","level":"INFO","fields":{"message":"Listening on 0.0.0.0:8080"},"target":"catscii"}
^C^C^C

And now it listens on some port! Unfortunately:

That port is only accessible from within the Docker container right now
Doing "Ctrl-C" to kill the container doesn't work.

So first off, let's kill our running container:

Shell session

$ docker ps
CONTAINER ID   IMAGE     COMMAND          CREATED              STATUS              PORTS     NAMES
d09a63ca7640   catscii   "/app/catscii"   About a minute ago   Up About a minute             flamboyant_chaplygin

# note: unique prefixes work here
$ docker kill d09
d09

Ctrl-C doesn't exit my docker container

If you look up "Ctrl-C docker not worky send help" online, you'll find tons of advice that doesn't work.

Ctrl+C from a Linux shell, sends the "INT" (interrupt) signal, and the default handler for that exits the process. It certainly works outside the container:

Shell session

$ cargo run --quiet
{"timestamp":"2022-12-23T16:56:47.798601Z","level":"INFO","fields":{"message":"Creating honey client","log.target":"libhoney::client","log.module_path":"libhoney::client","log.file":"/home/amos/.cargo/git/checkouts/libhoney-rust-3b5a30a6076b74c0/9871051/src/client.rs","log.line":78},"target":"libhoney::client"}
{"timestamp":"2022-12-23T16:56:47.798708Z","level":"INFO","fields":{"message":"transmission starting","log.target":"libhoney::transmission","log.module_path":"libhoney::transmission","log.file":"/home/amos/.cargo/git/checkouts/libhoney-rust-3b5a30a6076b74c0/9871051/src/transmission.rs","log.line":124},"target":"libhoney::transmission"}
{"timestamp":"2022-12-23T16:56:47.846784Z","level":"INFO","fields":{"message":"Listening on 0.0.0.0:8080"},"target":"catscii"}
^C

The explanation lies in the Docker docs:

A process running as PID 1 inside a container is treated specially by Linux: it ignores any signal with the default action. As a result, the process will not terminate on SIGINT or SIGTERM unless it is coded to do so.

Docker run reference

We could add an explicit handler for SIGINT with something like ctrlc, and it'd work. Or we could simply have another process as PID 1, like /bin/sh:

Dockerfile

# quick and dirty fix, consider not doing that:
CMD ["/bin/sh", "-c", "/app/catscii"]

But because I feel bad even suggesting it, let's implement "the proper way" instead.

Implementing graceful shutdown

Not only will we handle SIGINT, we'll also make it so it does graceful shutdown, which means it'll finish handling in-flight requests before stopping.

We don't actually need the ctrlc crate, because tokio provides an asynchronous way to deal with signals.

The end of our main function becomes:

Rust code

// in `src/main.rs`

#[tokio::main]
async fn main() {
    // omitted: all sorts of setup

    let quit_sig = async {
        _ = tokio::signal::ctrl_c().await;
        warn!("Initiating graceful shutdown");
    };

    let addr = "0.0.0.0:8080".parse().unwrap();
    info!("Listening on {addr}");
    axum::Server::bind(&addr)
        .serve(app.into_make_service())
        // new!                   👇
        .with_graceful_shutdown(quit_sig)
        .await
        .unwrap();
}

That's all it takes!

Let's not forget to rebuild the app:

Shell session

$ DOCKER_BUILDKIT=1 docker build --tag catscii .
(cut)

And then we can run it, publishing port 8080 so we can actually reach it:

Shell session

$ docker run -it --env SENTRY_DSN --env HONEYCOMB_API_KEY -p 8080:8080/tcp --rm catscii
{"timestamp":"2022-12-23T17:08:26.759656Z","level":"INFO","fields":{"message":"Creating honey client","log.target":"libhoney::client","log.module_path":"libhoney::client","log.file":"/root/.cargo/git/checkouts/libhoney-rust-3b5a30a6076b74c0/9871051/src/client.rs","log.line":78},"target":"libhoney::client"}
{"timestamp":"2022-12-23T17:08:26.759679Z","level":"INFO","fields":{"message":"transmission starting","log.target":"libhoney::transmission","log.module_path":"libhoney::transmission","log.file":"/root/.cargo/git/checkouts/libhoney-rust-3b5a30a6076b74c0/9871051/src/transmission.rs","log.line":124},"target":"libhoney::transmission"}
{"timestamp":"2022-12-23T17:08:26.763730Z","level":"INFO","fields":{"message":"Listening on 0.0.0.0:8080"},"target":"catscii"}

(the app keeps running)

And then, from another terminal (CmdOrCtrl+Shift+Backtick in VS Code, or a plus button somewhere to the top-right of the Terminal pane), we can reach it:

Shell session

$ curl -I localhost:8080
HTTP/1.1 200 OK
content-type: text/html; charset=utf-8
content-length: 134810
date: Fri, 23 Dec 2022 17:09:23 GMT

And, thanks to VS Code's automated port forwarding functionality, we can even access it in a local browser!

Cool bear's hot tip

Sometimes VS Code's automated port forwarding will get in an inconsistent state (especially if your host computer's gone to sleep a couple times and it had to reconnect to the VM over SSH).

Using the "Reload Window" command from the command palette ("F1" to open) usually sorts this out.

So, mission accomplished?

Not really. We want to run this in production!

This article is part 5 of the Building a Rust service with Nix series.

Read the next part

Discuss this article on /r/fasterthanlime

If you liked what you saw, please support my work!

Donate on GitHub Donate on Patreon

Looking for the homepage?

Another article: Pin and suffering

Writing a Dockerfile for catscii

Contents