Faster Docker builds with uv, pipenv, poetry, or pip-tools

Docker builds can be slow, and waiting for a build to finish is probably not how you want to spend your time. So you want faster builds—and caching is a great way to get there.

If the files you’re relying on haven’t changed, the Docker build can reuse previously cached layers for this particular build. And so if you separate out installation of dependencies from installation of your code in your Dockerfile you’ll usually get faster builds: if just your code changes, you won’t have to wait for all dependencies to be installed when you rebuild the Docker image.

In this article I’ll cover:

1. An example of how to get slow builds.
2. Faster builds by installing requirements first, from a requirements.txt file.
3. Managing your requirements.txt with pip-tools.
4. Managing your requirements.txt with uv.
5. Using pipenv in your Docker build.
6. Using poetry in your Docker build.

How to get slow builds

A common way of listing your dependencies is to put them in pyproject.toml:

# ...

[project]
dependencies = [
  "httpx",
  "flask",
]

# ...

Your Dockerfile might then look like this:

FROM python:3.11
COPY . /tmp/myapp
RUN pip install /tmp/myapp
CMD flask run exampleapp:app

Note: Outside any specific best practice being demonstrated, the Dockerfiles in this article are not examples of best practices, since the added complexity would obscure the main point of the article.

Need to ship quickly, and don’t have time to figure out every detail on your own? Read the concise, action-oriented Python on Docker Production Handbook.

The problem with this setup is that every time you change the code, that invalidates the COPY . /tmp/myapp layer in the Docker cache, as well as all subsequent lines in the Dockerfile. And so every time you rebuild the image, you will need to reinstall the dependencies and your code.

Another problem: reproducible builds

The scheme above has an additional problem: it always installs the latest version of the dependencies. So if you build the image on a computer that doesn’t have the cache populated, you might get a different set of installed dependencies than what would get built if you did have the Docker cache populated.

In short, your builds aren’t reproducible.

And that can lead to a variety of problems, e.g. random breakage when packages get upgraded without your knowledge, or hard-to-reproduce differences in behavior (“but it worked on my computer!”).

The solution: two kinds of dependencies in two different files

To solve this you want to keep two versions of your dependencies:

1. The logical dependencies of your application, i.e. the packages you directly import. In our example, flask and dateutil.
2. The pinned transitive dependencies. That is, all of flask’s dependencies (and their dependencies, and so on), pinned to a particular version.

The logical dependencies can be used to regenerate the pinned dependencies on the demand. The pinned dependencies are what you use to get reproducible builds. And in our Docker build we only need to copy in the file with pinned dependencies.

Reproducible builds using pip-tools

pip-tools is one easy way to do this. Given we have the logical dependencies (httpx and flask) in pyproject.toml, we can pip install pip-tools and then run (with matching Python version and ideally operating system):

$ pip-compile pyproject.toml -o requirements.txt

And the resulting requirements.txt looks like this:

#
# This file is autogenerated by pip-compile with Python 3.12
# by the following command:
#
#    pip-compile --output-file=requirements.txt pyproject.toml
#
anyio==4.8.0
    # via httpx
blinker==1.9.0
    # via flask
certifi==2024.12.14
    # via
    #   httpcore
    #   httpx
# ... and so on ...

You check in both pyproject.toml and requirements.txt into version control. And now are Dockerfile looks like this:

FROM python:3.12
COPY requirements.txt /tmp
RUN pip install -r /tmp/requirements.txt
COPY . /tmp/myapp
RUN pip install /tmp/myapp
CMD flask run exampleapp:app

Notice that initially we only copy requirements.txt, so that changes to the code won’t invalidate the caching at this point.

Reproducible builds with uv

The uv packaging tool re-implements pip and pip compile, only faster (and much more besides!). To use it as a replacement for pip compile, you just install it, then:

$ uv pip compile pyproject.toml -o requirements.txt

The main difference most of the time is that it will run much faster than pip-compile.

Fast reproducible Docker builds with poetry

Poetry is another tool that lets you manage logical and pinned dependencies, and much more besides. If you’re already using Poetry, you can use the export plugin for Poetry to export dependencies to a requirements.txt file, and then your Dockerfile doesn’t need to use poetry at all.

First, install the plugin. Then:

$ poetry export -f requirements.txt -o requirements.txt

Alternatively, you can write a Dockerfile that first installs only dependencies, and later installs the actual application code. See this article for details.

Fast reproducible Docker builds with pipenv

pipenv is another tool that allows you to maintain logical dependencies (in a Pipfile) and pinned dependencies (in a Pipfile.lock). It also does a whole lot more, e.g. virtualenv management.

Much of what it does isn’t relevant to building Docker images, though, so the easy way to use it in your Docker build is to export a requirements.txt file. You can do this outside your Docker build, and just commit the resulting file to version control and use the Dockerfile above:

$ pipenv requirements > requirements.txt

This scheme ensures that your Dockerfile doesn’t need to know anything about pipenv. This does require you to remember to regenerate requirements.txt every time you update Pipfile.lock.

Alternatively, you can do the export in the build itself:

FROM python:3.12
RUN pip install pipenv
COPY Pipfile* /tmp
RUN cd /tmp && pipenv requirements > requirements.txt
RUN pip install -r /tmp/requirements.txt
COPY . /tmp/myapp
RUN pip install /tmp/myapp
CMD flask run exampleapp:app

Note that a better setup, omitted for clarity, would have you install pipenv in such a way that its dependencies don’t impact your code, e.g. by using a virtualenv for your code.

The takeaway

To get fast, reproducible builds for your application:

1. Separate logical dependencies and pinned dependencies (using pip-tools, uv, pipenv, or poetry; pdm is another option).
2. Copy the pinned dependencies file separately and earlier in your Dockerfile, and then install dependencies using it before copying in the bulk of the code, to ensure faster builds.