Build secrets in Docker and Compose v1, the secure way

by
Last updated 21 Mar 2023, originally created 30 Mar 2020

When you’re building a Docker image, you might need to use some secrets: the password to a private package repository, for example. You don’t want this secret to end up embedded in the image, because then anyone who somehow gets access to the image will get access to your private repository.

Note: If you’re thinking “why not just use environment variables?”, those are used for runtime secrets once the image is built. This article is about build secrets, which are used while building the image via a Dockerfile.

Docker Compose is undergoing a v2 rewrite, and it now has support for build secrets. You can see an example in the pull request, and the syntax is described in the reference Be aware that the v2 rewrite uses a slightly different configuration language than previous versions of Compose.

Docker Compose v1 series is now deprecated, and will no longer be maintained after June 2023, so secret support will never be added. So if you’re still using v1, this article will demonstrate a technique that allows you use the same Dockerfile to both build production images securely with secrets, while still enabling easy development with Docker Compose.

Two use cases for your Dockerfile

It’s handy to use the same Dockerfile both for production use, and for local development with Docker Compose. Typically you’d use your Dockerfile with Compose’s build functionality:

version: "3.7"
services:
  yourapp:
    build:
      context: "."

You can then do:

$ docker-compose up

And it will (re)build the image for you, and then run it.

For production use, you build the image and then push it:

$ docker build -t myimage .
$ docker push myimage

So far, so good. But what if you need to build with a secret of some sort?

A first, insecure attempt

Let’s say we have a script that needs a build secret, for example to download a Python package from a private DevPI repository. For simplicity, we’ll just have our use_secret.sh script print the secret, to prove we have it:

#!/bin/bash
set -euo pipefail

echo "Secret is: $THEPASSWORD"

The naive way to pass in secrets is using Docker build args, since they’re supported everywhere, including Docker Compose.

Note: Outside any specific best practice being demonstrated, the Dockerfiles in this article are not examples of best practices, since the added complexity would obscure the main point of the article.

Python on Docker Production Handbook Need to ship quickly, and don’t have time to figure out every detail on your own? Read the concise, action-oriented Python on Docker Production Handbook.

FROM python:3.8-slim-buster
# Using ARG for build secrets is INSECURE!
ARG THEPASSWORD
COPY use_secret.sh .
RUN ./use_secret.sh

We can write a docker-compose.yml that passes in the secret:

version: "3.7"
services:
  yourapp:
    build:
      context: "."
      args:
        THEPASSWORD: "s3kr!t"

For local development we can run or build the image via Compose:

$ docker-compose build | grep Secret
Secret is: s3kr!t

And that’s fine.

We can also build the image using Docker, as preparation for pushing it to an image registry:

$ docker build -t myimage --build-arg THEPASSWORD=s3krit . | grep Secret
Secret is: s3krit

This is insecure: don’t ever do this. If we inspect the image layers, we’ll see that the secret has been embedded into the image!

$ docker history myimage
IMAGE               CREATED              CREATED BY                                      SIZE
c224231ec30b        47 seconds ago       |1 THEPASSWORD=s3krit /bin/sh -c ./use_secre…   0B
6aef62acf0db        48 seconds ago       /bin/sh -c #(nop) COPY file:7aa28bbe6595e0d5…   62B
f88b19ca8e65        About a minute ago   /bin/sh -c #(nop)  ARG THEPASSWORD              0B
...

Anyone who gets access to this image will know your password! What to do?

BuildKit secrets, a partial solution

BuildKit is a new and improved implementation of Docker image building. And Docker Compose has experimental support for using BuildKit as of v1.25.

But there’s a problem: Docker Compose doesn’t yet support the secrets feature in BuildKit. There’s an in-progress PR, but as of April 2021 it hasn’t been merged, let alone released.

So what we’re going to do is combine the two approaches:

  1. Docker Compose will continue to use build args to pass in secrets.
  2. For the production image, built with docker build, we’ll use BuildKit to pass in secrets.

This will allow us to use the same Dockerfile for both local development and for our final production image.

The combo approach

The way BuildKit secrets work is that a file with the secret gets mounted to a temporary location during the RUN command, e.g. /run/secrets/thepassword. Since it’s only mounted during a particular RUN command being run, it doesn’t end up embedded in the final image.

We modify use_secret.sh so that it checks if that file exists. If it does, it uses it to set the $THEPASSWORD environment variable; if the file doesn’t exist, we fall back to the environment variable. That means $THEPASSWORD can be set either by BuildKit secrets or via a build arg:

#!/bin/bash
set -euo pipefail
if [ -f /run/secrets/thepassword ]; then
   export THEPASSWORD=$(cat /run/secrets/thepassword)
fi

echo "Secret is: $THEPASSWORD"

Next, we modify the Dockerfile to enable BuildKit and mount the secret:

# syntax = docker/dockerfile:1.3
FROM python:3.8-slim-buster
# Only use the build arg for local development:
ARG THEPASSWORD
COPY use_secret.sh .
# Mount the secret to /run/secrets:
RUN --mount=type=secret,id=thepassword ./use_secret.sh

The docker-compose.yml file remains unchanged:

version: "3.7"
services:
  yourapp:
    build:
      context: "."
      args:
        THEPASSWORD: "s3kr!t"

Now we need to set two environment variables, one to tell Docker to use BuildKit, one to tell Compose to use the CLI version of Docker and therefore BuildKit. We also write the secret to a file:

$ export DOCKER_BUILDKIT=1
$ export COMPOSE_DOCKER_CLI_BUILD=1
$ echo 's3krit' > /tmp/mypassword

With Compose, the build arg is used:

$ docker-compose build --progress=plain \
    --no-cache 2>&1 | grep Secret
#12 0.347 Secret is: s3kr!t

Note that the --no-cache is just to ensure it actually rebuilds, in case you’ve run all the above yourself; in real usage you can omit it. The 2>&1 redirects stderr to stdout for easier grepping.

When we’re ready to build for production use, we use docker build with the BuildKit secrets feature:

$ docker build --no-cache -t myimage \
    --secret id=thepassword,src=/tmp/mypassword \
    --progress=plain . 2>&1 | grep Secret
#12 0.359 Secret is: s3krit

Is it secure?

Let’s make sure our secret wasn’t leaked:

$ docker history myimage
IMAGE               CREATED             CREATED BY                                      SIZE
a77f3c32b723        25 seconds ago      RUN |1 THEPASSWORD= /bin/sh -c ./use_secret.…   0B
<missing>           25 seconds ago      COPY use_secret.sh . # buildkit                 160B
...

Success! We passed in the secret to the same Dockerfile using both Compose and docker build, and in the latter case we didn’t leak the build secret.