Docker BuildKit: faster builds, new features, and now it’s stable
Building Docker images can be slow, and Docker’s build system is also missing some critical security features, in particular the ability to use build secrets without leaking them. So over the past few years the Docker developers have been working on a new backend for building images, BuildKit.
With the release of Docker 20.10 in late 2020, BuildKit is finally marked as stable–and you don’t need to upgrade to use it, you can use it with existing Docker 19.03 installs. And you might already be using it if you’re on macOS or Windows.
In this article you’ll learn:
- Some of the new features BuildKit adds.
- Some of the caveats, and corresponding workarounds.
- How to use BuildKit on Docker 19.03 and 20.10.
BuildKit’s new features
BuildKit has quite a few new features; here I’ll just mention some of them.
Faster builds using parallelism
Consider the following multi-stage
By building in multiple stages, it enables both caching for fast rebuilds and smaller images in production.
If you’re not familiar with the concept, start with part 1 of my 3-part series on multi-stage Docker builds in Python.
FROM python:3.8-slim-buster AS build-stage RUN apt-get update && apt-get install -y --no-install-recommends gcc RUN python -m venv /venv ENV PATH=/venv/bin:$PATH RUN pip install pyrsistent FROM python:3.8-slim-buster AS runtime-stage RUN apt-get update && apt-get -y upgrade COPY --from=build-stage /venv /venv ENV PATH=/venv/bin:$PATH ENTRYPOINT ["python", "-c", "import pyrsistent; print(pyrsistent.__file__)"]
Note: Outside the very specific topic under discussion, the Dockerfiles in this article are not examples of best practices, since the added complexity would obscure the main point of the article.
To ensure you’re following all the best practices you need to have a secure, correct, fast Dockerfiles, check out the Python on Docker Production Handbook.
On my computer, building this takes about 22 seconds with classic Docker. When I turn on BuildKit, however, it takes only 16 seconds.
This is because BuildKit can build multiple stages in parallel.
Notice that the second stage image’s
apt-get does not depend in any way on the first stage; the dependency happens only once the
COPY --from=build-stage happens.
BuildKit can figure that out and run the build steps in parallel until that dependency becomes a blocker.
Sometimes you need some secret or password to run your build, for example the password to private package repository. In classic Docker builds there is no good way to do this; the obvious methods are insecure, and the workarounds are hacky.
Note: It’s easy to confuse build secrets with runtime secrets. Here I am specifically talking about secrets that are only necessary when building the image, not secrets used by the running application.
BuildKit adds support for securely passing build secrets, as well as forwarding SSH authentication agent from the host into the Docker build. You can learn more in the somewhat out-of-date Docker docs, or read my article on BuildKit build secrets and how to use them with Compose. As we’ll see later on, Compose support is something of an annoyance with BuildKit.
BuildKit has many other new
Dockerfile features, allow you to:
- Have a filesystem cache for builds.
- Bind mount other images or stages into your build.
- Add an in-memory filesystem,
- and more.
You can see a full-list in the
docker/dockerfile image docs.
We’ll talk about this in the next section, and then give usage examples later on.
In classic Docker, the only way to get a new
Dockerfile feature was to upgrade to a new version of Docker.
For example, Docker 17.09 added the
COPY --chown option, but until you upgraded you couldn’t use it.
With BuildKit, the code that reads the
Dockerfile and issues the appropriate command–known as the “frontend”–can be specified and downloaded at build time.
This means you can always get the latest features–stable or experimental–without having to upgrade your Docker daemon.
The BuildKit frontend is distributed as a Docker image, specifically
Docker 20.10 includes a new stable
docker image buildx command, a replacement for the classic
docker image build command.
It supports things like multi-platform image building, and building multiple images concurrently to take advantage of shared parallelism.
You can learn more here, although as is often the case with Docker many of the new features aren’t very well documented. Also, at the time of writing the docs are still written for Docker 19.03 when this feature was still experimental.
There are two parts to using BuildKit: enabling it in a specific build, and choosing the “frontend” to use in your
Note that I’m assuming you’re using Docker 19.03 or later.
If you just want the short version:
- Set the
DOCKER_BUILDKITenvironment variable to
# syntax=docker/dockerfile:1.2as the first line of your
If you’re interested in more details, read the rest of this section.
Enabling BuildKit in your build
Enabling BuildKit depends on the version of Docker you’re using, and the platform you’re using.
If you’re using Docker Desktop on macOS or Windows:
- If you’ve newly installed it since October 2020, or have reset to factory defaults, BuildKit will be enabled by default for all builds.
- You can turn it on/off for all builds in Preferences > Docker Engine.
If it’s not on by default, for example on Linux, you will need to set the environment variable
$ export DOCKER_BUILDKIT=1
Enabling the latest BuildKit in your Dockerfile
As mentioned previously, BuildKit has a concept of a “frontend”, some code that parses the
Different versions of Docker ship with different versions of this frontend, but you can specify a version explicitly.
Docker 19.03 ships with a version that has none of the new BuildKit features enabled, and moreover it’s rather old and out of date, lacking many bugfixes. So you’ll want to specify a version explicitly.
Docker 20.10 ships with the
1.2 frontend, but you can still specify a specific version if you want.
In practice, you may as well, so that the
Dockerfile works with Docker 19.03 as well, and also so you can get the latest frontend version with the latest bugfixes without having to upgrade the whole Docker daemon.
The way you specify the frontend version is by adding a line to the top of your Dockerfile, basically a pointer to a Docker image:
# syntax=docker/dockerfile:1.2 FROM python:3.9-slim-buster # ...
You can also specify a specific stable version (e.g.
# syntax=docker/dockerfile:1.2.1) or the experimental version that has more features (
docker/dockerfile docs for more details.
Note: You will see on the web and even in Docker documentation references to older versions of
docker/dockerfile:1.0-experimental. Don’t use those versions, you want to use the stable
Problems and workarounds
As with any change, there are some problems with switching to BuildKit.
Classic Docker builds will print the build output as it runs.
BuildKit hides the output of successful commands once they’re done running.
You can get output that’s closer to classic Docker by using the
$ DOCKER_BUILDKIT=1 docker image build --progress=plain . ...
Docker Compose doesn’t work out of the box with BuildKit. You can enable support for BuildKit by setting an appropriate environment variable:
$ export DOCKER_BUILDKIT=1 $ export COMPOSE_DOCKER_CLI_BUILD=1
Unfortunately this method of using Docker results in somewhat worse error messages when builds fail. In addition, some BuildKit features aren’t supported via Compose’s configuration language; see my article on using BuildKit secrets with Compose for a workaround for one missing feature.
More difficult debugging of failed builds
In classic Docker, every step of the build results in a new image, accessible via the ID reported in the build’s output. That meant when builds failed, it was easy to run a container off intermediate steps. In BuildKit, this is no longer possible; writing out each intermediate step was felt to be a performance bottleneck.
Instead, if you want to start a container off an early step in the build, just comment out the later steps of the
Dockerfile and build that.
Not quite as fast or elegant, but caching will ensure it happens pretty quickly.
Incompatibility with Podman
Some of the new BuildKit features are optimizations, like parallel builds.
Others are new
There are a number of other projects that support building
Some of them already use BuildKit, so using these new features is not a problem.
But Podman, RedHat’s reimplemented-from-scratch Docker, will not support these features at the moment.
Time to switch?
While BuildKit does introduce some new problems, it also introduces many new enhancements and performance benefits.
My personal experience running the test suite for my Docker template for Python is that the newer versions are quite backwards-compatible in how they interpret
Older versions of BuildKit didn’t work quite as well, it’s definitely improved.
I have heard some people complain their linters didn’t run, perhaps due to BuildKit being smart enough to skip unused stages in multi-stage builds, unlike classic Docker.
Given BuildKit is enabled by default on new macOS and Windows installs, at the very least you should make sure your Docker build works correctly on BuildKit. But unless some of the caveats significantly impact you, for complex projects it seems worth switching now.
Learn how to build fast, production-ready Docker images—read the rest of the Docker packaging guide for Python.
Production Docker packaging is too complicated to learn from Google searches
With as much as a dozen different intersecting technologies, and an unknown number of details to get right, Docker packaging isn't simple, especially for production.
But you still need fast builds that save you time, and security best practices that keep you safe.
Take the fast path to learning best practices, by using the Python on Docker Production Handbook.
Free ebook: Introduction to Dockerizing for Production
Learn a step-by-step iterative DevOps packaging process in this free mini-ebook. You'll learn what to prioritize, the decisions you need to make, and the ongoing organizational processes you need to start.
Plus, you'll join my newsletter and get weekly articles covering practical tools and techniques, from Docker packaging to Python best practices.