Docker BuildKit: faster builds, new features, and now it’s stable

by
Last updated 21 Mar 2023, originally created 11 Jan 2021

Building Docker images can be slow, and Docker’s build system is also missing some critical security features, in particular the ability to use build secrets without leaking them. So over the past few years the Docker developers have been working on a new backend for building images, BuildKit.

Since the release of Docker 20.10, BuildKit is now stable. You’re probably already using it if you’re on macOS or Windows, and starting with version 23.0 (released in early 2023) it will be enabled by default on Linux as well.

In this article you’ll learn:

  • Some of the new features BuildKit adds (faster builds! secrets!).
  • Some of the caveats, and corresponding workarounds.
  • How to use BuildKit on Docker 19.03 and 20.10 if it’s not on by default.

BuildKit’s new features

BuildKit has quite a few new features; here I’ll just mention some of them.

Faster builds using parallelism

Consider the following multi-stage Dockerfile. By building in multiple stages, it enables both caching for fast rebuilds and smaller images in production. If you’re not familiar with the concept, start with part 1 of my 3-part series on multi-stage Docker builds in Python.

FROM python:3.8-slim-bullseye AS build-stage
RUN apt-get update && apt-get install -y --no-install-recommends gcc
RUN python -m venv /venv
ENV PATH=/venv/bin:$PATH
RUN pip install pyrsistent

FROM python:3.8-slim-bullseye AS runtime-stage
RUN apt-get update && apt-get -y upgrade
COPY --from=build-stage /venv /venv
ENV PATH=/venv/bin:$PATH
ENTRYPOINT ["python", "-c", "import pyrsistent; print(pyrsistent.__file__)"]

Note: Outside any specific best practice being demonstrated, the Dockerfiles in this article are not examples of best practices, since the added complexity would obscure the main point of the article.

Python on Docker Production Handbook Need to ship quickly, and don’t have time to figure out every detail on your own? Read the concise, action-oriented Python on Docker Production Handbook.

On my computer, building this takes about 22 seconds with classic Docker. When I turn on BuildKit, however, it takes only 16 seconds.

This is because BuildKit can build multiple stages in parallel. Notice that the second stage image’s apt-get does not depend in any way on the first stage; the dependency happens only once the COPY --from=build-stage happens. BuildKit can figure that out and run the build steps in parallel until that dependency becomes a blocker.

Build secrets

Sometimes you need some secret or password to run your build, for example the password to private package repository. In classic Docker builds there is no good way to do this; the obvious methods are insecure, and the workarounds are hacky.

Note: It’s easy to confuse build secrets with runtime secrets. Here I am specifically talking about secrets that are only necessary when building the image, not secrets used by the running application.

BuildKit adds support for securely passing build secrets, as well as forwarding SSH authentication agent from the host into the Docker build. You can learn more in the somewhat out-of-date Docker docs, or read my article on BuildKit build secrets and how to use them with Compose. As we’ll see later on, Compose support is something of an annoyance with BuildKit.

Other Dockerfile features

BuildKit has many other new Dockerfile features, allow you to:

  • Have a filesystem cache for builds.
  • Bind mount other images or stages into your build.
  • Add an in-memory filesystem,
  • and more.

You can see a full-list in the docker/dockerfile image docs. What is docker/dockerfile? We’ll talk about this in the next section, and then give usage examples later on.

Upgradability

In classic Docker, the only way to get a new Dockerfile feature was to upgrade to a new version of Docker. For example, Docker 17.09 added the COPY --chown option, but until you upgraded you couldn’t use it.

With BuildKit, the code that reads the Dockerfile and issues the appropriate command–known as the “frontend”–can be specified and downloaded at build time. This means you can always get the latest features–stable or experimental–without having to upgrade your Docker daemon. The BuildKit frontend is distributed as a Docker image, specifically docker/dockerfile.

More features

Docker 20.10 includes a new stable docker image buildx command, a replacement for the classic docker build/docker image build command. It supports things like multi-platform image building, and building multiple images concurrently to take advantage of shared parallelism.

You can learn more here, although as is often the case with Docker many of the new features aren’t very well documented.

Using BuildKit

There are two parts to using BuildKit: enabling it in a specific build, and choosing the “frontend” to use in your Dockerfile. Note that I’m assuming you’re using Docker 19.03 or later.

If you just want the short version:

  • Set the DOCKER_BUILDKIT environment variable to 1.
  • Add # syntax=docker/dockerfile:1.5 as the first line of your Dockerfile.

If you’re interested in more details, read the rest of this section.

Enabling BuildKit in your build

Enabling BuildKit depends on the version of Docker you’re using, and the platform you’re using.

If you’re using Docker Desktop on macOS or Windows:

  • If you’ve newly installed it since October 2020, or have reset to factory defaults, BuildKit will be enabled by default for all builds.
  • You can turn it on/off for all builds in Preferences > Docker Engine.

If it’s not on by default, for example on Linux, you will need to set the environment variable DOCKER_BUILDKIT to 1, e.g.:

$ export DOCKER_BUILDKIT=1

Enabling the latest BuildKit in your Dockerfile

As mentioned previously, BuildKit has a concept of a “frontend”, some code that parses the Dockerfile. Different versions of Docker ship with different versions of this frontend, but you can specify a version explicitly.

Docker 19.03 ships with a version that has none of the new BuildKit features enabled, and moreover it’s rather old and out of date, lacking many bugfixes. So you’ll want to specify a version explicitly.

Docker 20.10 ships with the 1.2 frontend, but you can still specify a specific version if you want. In practice, you may as well, so that the Dockerfile works with Docker 19.03 as well, and also so you can get the latest frontend version with the latest bugfixes without having to upgrade the whole Docker daemon.

The way you specify the frontend version is by adding a line to the top of your Dockerfile, basically a pointer to a Docker image:

# syntax=docker/dockerfile:1.5
FROM python:3.9-slim-bullseye
# ...

You can also specify a specific stable version (e.g. # syntax=docker/dockerfile:1.5.2) or the experimental version that has more features (# syntax=docker/dockerfile:1.5-labs). See the docker/dockerfile docs for more details.

Note: You will see on the web and even in Docker documentation references to older versions of docker/dockerfile, e.g. docker/dockerfile:1.0-experimental. Don’t use those versions, you want to use the stable 1.3.

Problems and workarounds

As with any change, there are some problems with switching to BuildKit.

Hidden output

Classic Docker builds will print the build output as it runs. BuildKit hides the output of successful commands once they’re done running. You can get output that’s closer to classic Docker by using the --progress=plain option:

$ DOCKER_BUILDKIT=1 docker image build --progress=plain .
...

Docker Compose

Docker Compose v1 doesn’t work out of the box with BuildKit. You can enable support for BuildKit by setting an appropriate environment variable:

$ export DOCKER_BUILDKIT=1
$ export COMPOSE_DOCKER_CLI_BUILD=1

Docker Compose v2 is designed to work with BuildKit.

More difficult debugging of failed builds

In classic Docker, every step of the build results in a new image, accessible via the ID reported in the build’s output. That meant when builds failed, it was easy to run a container off intermediate steps. In BuildKit, this is no longer possible; writing out each intermediate step was felt to be a performance bottleneck.

Instead, if you want to start a container off an early step in the build, just comment out the later steps of the Dockerfile and build that. Not quite as fast or elegant, but caching will ensure it happens pretty quickly.

Incompatibility with Podman (though not as much as in the past)

Some of the new BuildKit features are optimizations, like parallel builds. Others are new Dockerfile features.

There are a number of other projects that support building Dockerfiles. Some of them already use BuildKit, so using these new features is not a problem.

Podman, RedHat’s reimplemented-from-scratch Docker, will not support all of these features at the moment. They have been doing a pretty good job of catching up, however, so in practice you might be fine.

Time to switch

Starting with version 23.0 (released February 2023), BuildKit is the default not just on macOS and Windows, but on Linux as well. Stable Linux distributions won’t have this version for a while until you install it manually, but it’s just a matter of time before BuildKit is on by default everywhere. As such, you can and should start taking advantage of its functionality unless you are using some other tool to build images that doesn’t support its extended functionality.