Docker BuildKit: faster builds, new features, and now it’s stable

Building Docker images can be slow, and Docker’s build system is also missing some critical security features, in particular the ability to use build secrets without leaking them. So over the past few years the Docker developers have been working on a new backend for building images, BuildKit.

With the release of Docker 20.10 in late 2020, BuildKit is finally marked as stable–and you don’t need to upgrade to use it, you can use it with existing Docker 19.03 installs. And you might already be using it if you’re on macOS or Windows.

In this article you’ll learn:

  • Some of the new features BuildKit adds.
  • Some of the caveats, and corresponding workarounds.
  • How to use BuildKit on Docker 19.03 and 20.10.

BuildKit’s new features

BuildKit has quite a few new features; here I’ll just mention some of them.

Faster builds using parallelism

Consider the following multi-stage Dockerfile. By building in multiple stages, it enables both caching for fast rebuilds and smaller images in production. If you’re not familiar with the concept, start with part 1 of my 3-part series on multi-stage Docker builds in Python.

FROM python:3.8-slim-buster AS build-stage
RUN apt-get update && apt-get install -y --no-install-recommends gcc
RUN python -m venv /venv
ENV PATH=/venv/bin:$PATH
RUN pip install pyrsistent

FROM python:3.8-slim-buster AS runtime-stage
RUN apt-get update && apt-get -y upgrade
COPY --from=build-stage /venv /venv
ENV PATH=/venv/bin:$PATH
ENTRYPOINT ["python", "-c", "import pyrsistent; print(pyrsistent.__file__)"]

Note: Outside the very specific topic under discussion, the Dockerfiles in this article are not examples of best practices, since the added complexity would obscure the main point of the article.

To ensure you’re writing secure, correct, fast Dockerfiles, consider my Python on Docker Production Handbook, which includes a packaging process and >70 best practices.

On my computer, building this takes about 22 seconds with classic Docker. When I turn on BuildKit, however, it takes only 16 seconds.

This is because BuildKit can build multiple stages in parallel. Notice that the second stage image’s apt-get does not depend in any way on the first stage; the dependency happens only once the COPY --from=build-stage happens. BuildKit can figure that out and run the build steps in parallel until that dependency becomes a blocker.

Build secrets

Sometimes you need some secret or password to run your build, for example the password to private package repository. In classic Docker builds there is no good way to do this; the obvious methods are insecure, and the workarounds are hacky.

Note: It’s easy to confuse build secrets with runtime secrets. Here I am specifically talking about secrets that are only necessary when building the image, not secrets used by the running application.

BuildKit adds support for securely passing build secrets, as well as forwarding SSH authentication agent from the host into the Docker build. You can learn more in the somewhat out-of-date Docker docs, or read my article on BuildKit build secrets and how to use them with Compose. As we’ll see later on, Compose support is something of an annoyance with BuildKit.

Other Dockerfile features

BuildKit has many other new Dockerfile features, allow you to:

  • Have a filesystem cache for builds.
  • Bind mount other images or stages into your build.
  • Add an in-memory filesystem,
  • and more.

You can see a full-list in the docker/dockerfile image docs. What is docker/dockerfile? We’ll talk about this in the next section, and then give usage examples later on.

Upgradability

In classic Docker, the only way to get a new Dockerfile feature was to upgrade to a new version of Docker. For example, Docker 17.09 added the COPY --chown option, but until you upgraded you couldn’t use it.

With BuildKit, the code that reads the Dockerfile and issues the appropriate command–known as the “frontend”–can be specified and downloaded at build time. This means you can always get the latest features–stable or experimental–without having to upgrade your Docker daemon. The BuildKit frontend is distributed as a Docker image, specifically docker/dockerfile.

More features

Docker 20.10 includes a new stable docker image buildx command, a replacement for the classic docker build/docker image build command. It supports things like multi-platform image building, and building multiple images concurrently to take advantage of shared parallelism.

You can learn more here, although as is often the case with Docker many of the new features aren’t very well documented. Also, at the time of writing the docs are still written for Docker 19.03 when this feature was still experimental.

Using BuildKit

There are two parts to using BuildKit: enabling it in a specific build, and choosing the “frontend” to use in your Dockerfile. Note that I’m assuming you’re using Docker 19.03 or later.

If you just want the short version:

  • Set the DOCKER_BUILDKIT environment variable to 1.
  • Add # syntax=docker/dockerfile:1.2 as the first line of your Dockerfile.

If you’re interested in more details, read the rest of this section.

Enabling BuildKit in your build

Enabling BuildKit depends on the version of Docker you’re using, and the platform you’re using.

If you’re using Docker Desktop on macOS or Windows:

  • If you’ve newly installed it since October 2020, or have reset to factory defaults, BuildKit will be enabled by default for all builds.
  • You can turn it on/off for all builds in Preferences > Docker Engine.

If it’s not on by default, for example on Linux, you will need to set the environment variable DOCKER_BUILDKIT to 1, e.g.:

$ export DOCKER_BUILDKIT=1

Enabling the latest BuildKit in your Dockerfile

As mentioned previously, BuildKit has a concept of a “frontend”, some code that parses the Dockerfile. Different versions of Docker ship with different versions of this frontend, but you can specify a version explicitly.

Docker 19.03 ships with a version that has none of the new BuildKit features enabled, and moreover it’s rather old and out of date, lacking many bugfixes. So you’ll want to specify a version explicitly.

Docker 20.10 ships with the 1.2 frontend, but you can still specify a specific version if you want. In practice, you may as well, so that the Dockerfile works with Docker 19.03 as well, and also so you can get the latest frontend version with the latest bugfixes without having to upgrade the whole Docker daemon.

The way you specify the frontend version is by adding a line to the top of your Dockerfile, basically a pointer to a Docker image:

# syntax=docker/dockerfile:1.2
FROM python:3.9-slim-buster
# ...

You can also specify a specific stable version (e.g. # syntax=docker/dockerfile:1.2.1) or the experimental version that has more features (# syntax=docker/dockerfile:1.2-labs). See the docker/dockerfile docs for more details.

Note: You will see on the web and even in Docker documentation references to older versions of docker/dockerfile, e.g. docker/dockerfile:1.0-experimental. Don’t use those versions, you want to use the stable 1.2.

Problems and workarounds

As with any change, there are some problems with switching to BuildKit.

Hidden output

Classic Docker builds will print the build output as it runs. BuildKit hides the output of successful commands once they’re done running. You can get output that’s closer to classic Docker by using the --progress=plain option:

$ DOCKER_BUILDKIT=1 docker image build --progress=plain .
...

Docker Compose

Docker Compose doesn’t work out of the box with BuildKit. You can enable support for BuildKit by setting an appropriate environment variable:

$ export DOCKER_BUILDKIT=1
$ export COMPOSE_DOCKER_CLI_BUILD=1

Unfortunately this method of using Docker results in somewhat worse error messages when builds fail. In addition, some BuildKit features aren’t supported via Compose’s configuration language; see my article on using BuildKit secrets with Compose for a workaround for one missing feature.

More difficult debugging of failed builds

In classic Docker, every step of the build results in a new image, accessible via the ID reported in the build’s output. That meant when builds failed, it was easy to run a container off intermediate steps. In BuildKit, this is no longer possible; writing out each intermediate step was felt to be a performance bottleneck.

Instead, if you want to start a container off an early step in the build, just comment out the later steps of the Dockerfile and build that. Not quite as fast or elegant, but caching will ensure it happens pretty quickly.

Incompatibility with Podman

Some of the new BuildKit features are optimizations, like parallel builds. Others are new Dockerfile features.

There are a number of other projects that support building Dockerfiles. Some of them already use BuildKit, so using these new features is not a problem.

But Podman, RedHat’s reimplemented-from-scratch Docker, will not support these features at the moment.

Time to switch?

While BuildKit does introduce some new problems, it also introduces many new enhancements and performance benefits.

My personal experience running the test suite for my Docker template for Python is that the newer versions are quite backwards-compatible in how they interpret Dockerfiles. Older versions of BuildKit didn’t work quite as well, it’s definitely improved. I have heard some people complain their linters didn’t run, perhaps due to BuildKit being smart enough to skip unused stages in multi-stage builds, unlike classic Docker.

Given BuildKit is enabled by default on new macOS and Windows installs, at the very least you should make sure your Docker build works correctly on BuildKit. But unless some of the caveats significantly impact you, for complex projects it seems worth switching now.


Learn how to build fast, production-ready Docker images—read the rest of the Docker packaging guide for Python.