Important: The template is proprietary software, licensed under the terms of the software license. The short version: you canât redistribute it outside your organization, e.g. to customers or in open source projects.
If you have any questions or problems please email meâbut do please first read this document in detail to make sure itâs not already covered.
In addition, while I aim to support many common features, not everything will be supported out of the box in the template. Some options:
Before you start Dockerizing your application, you can try it out without touching your code. In the template directory, run:
$ mkdir /tmp/python-docker-test
$ python3 install.py /tmp/python-docker-test
... say 'y' at the prompt ...You now have a directory with everything needed to build a Docker
image. To build an image called exampleapp:
$ cd /tmp/python-docker-test
$ python3 docker-build/scripts/builder.py build exampleapp
Successfully finished building images:
exampleapp:nogit-build
exampleapp:nogit-runtimeBy default two images are built: an intermediate image tagged with
:<git-branch>-build where everything is built, and a
final, smaller image tagged with
:<git-branch>-runtime which is what youâll actually
run in production. In this case, you donât have a Git branch, so instead
it will use ânogitâ. To run the resulting image, then, youâll need to
do:
$ docker container run --rm exampleapp:nogit-runtime(You might need to sudo docker run on Linux.)
If all goes well, you should see the default entrypoint output,
telling you to edit docker-build/entrypoint.sh and listing
the installed packages.
Note: If you get the following problem:
File "docker-build/scripts/builder.py", line 2 SyntaxError: Non-ASCII character '\xe2' in file docker-build/scripts/builder.py on line 2, but no encoding declared; see http://python.org/dev/peps/pep-0263/ for detailsItâs because you used Python 2; make sure you use Python 3.9 or later.
Now itâs time to package your application!
This template includes not just a Dockerfile but also
the necessary build infrastructure. For example, thereâs the
need for ongoing rebuilds without caching to ensure all packages are
up-to-date with security fixes. At a high level, this template builds
Docker images as follows:
There are actually two images built: one image used to compile and build your application, and a runtime image which is what you will run in production. This helps keep your runtime images small and more secure, since they donât include extraâand unnecessaryâthings like compilers. See this article for details.
Letâs copy the necessary files in to your applicationâs repository.
First, create a new branch in your version control.
Second, run the following command in the directory where you unpacked the template:
$ python3 install.py /path/to/your/app/repository/
$ cd /path/to/your/app/repository/This will give you a list of the installed files. It will overwrite any existing files, which is why youâre doing this in a branch!
Third, commit the initial files to the branch.
By having an unmodified version committed to your repository, youâll have an easier time seeing what specific customizations you have made, and reverting back to the baseline if necessary.
Beyond this point youâll be editing files in your repository, not in the original template directory!*
Dockerfile: The main configuration for the Docker
image.By default the template using the python Docker base
images, pre-configured with a specific Python version. If you want to
use a different Python version, you need to change the
configuration.
If you want to use a different base image altogether, see the
comments in the top of the Dockerfile by the
FROM statement.
At the top of the Dockerfile, change:
ARG BUILD_PYTHON_VERSION="3.13"to the version you want, e.g.:
ARG BUILD_PYTHON_VERSION="3.11"The template supports installing Python dependencies based on a number of different configuration mechanisms:
requirements.txt file.uvâsâ uv.lock.pyproject.toml +
poetry.lock.Pipfile +
Pipfile.lock.setup.py/setup.cfg; this is not
recommended since it will slow down rebuilding images. If that is your
current mechanism, I recommend automatically creating a
requirements.txt from setup.py using
pip-tools.pyproject.toml-based build tools, like
Flit.The template assumes these configuration files are in the root
directory of your application. If you store these elsewhere, edit the
Dockerfileâs relevant COPY file to point to
the correct path.
First, you can check which of the two files will be used:
$ python3 docker-build/scripts/install-dependencies.py diagnoseSecond, build the image, and then check which packages were
installed. If youâre running off of a Git repository, replace
yourbranch below with the name of the current Git
branch:
$ python3 docker-build/scripts/builder.py build exampleapp
Successfully finished building images:
exampleapp:yourbranch-build
exampleapp:yourbranch-runtime
$ docker run exampleapp:yourbranch-build pip list
$ docker run exampleapp:yourbranch-runtime pip listDockerfile: The main configuration for the Docker
image.Some applications, expect that they will be installed via
pip install or poetry install, and then run
via the installed version. That is the default in this template.
However, some applications assume youâre running your code out of the directory where the original code sits, i.e. the code doesnât get installed. You will need to tweak the configuration to make this happen.
If your application is expected to run out of the directory where
itâs built, edit the top of Dockerfile and change:
ARG RUN_IN_PLACE=0To say:
ARG RUN_IN_PLACE=1If you also need to compile some code in-place, you might also need
to add relevant commands, right after the call to
install-code.py later in the Dockerfile. For
example:
RUN python setup.py build_ext -iDockerfile: The main configuration for the Docker
image.Sometimes you will need additional build steps in your
Dockerfile. For example:
Add those steps in the Dockerfile, specifically where it
says:
# If you need to run additional commands to ensure your code runs correctly, run
# them here.
#
# RUN python some-setup-script.pyTo minimize cache invalidation, try to follow the pattern you already
see in the Dockerfile of first copying in just enough of
the files to do the next build step. E.g. first copy in your package
dependencies list, install those packages, and then in a later step do
the actual build.
docker-build/entrypoint.sh: The script that is run when
someone runs your image via docker run or your deployment
system.When your Docker image is run (via docker run yourimage
or whatever your deployment environment is), it will run
docker-build/entrypoint.sh. So you will need to edit this
file to ensure it runs the correct command.
The default script just prints some debug output, so youâll need to change it to run whatever command or commands your application needs to start. Make sure:
exec so it exits
cleanly.0.0.0.0 rather than
127.0.0.1 (see here
for explanation).The script include a commented out example of a Gunicorn WSGI setup suitable for web applications.
You should now be able to run your image locally on your own
computer, so itâs time to try it out and see if it works. If youâre
running off of a Git repository, replace yourbranch below
with the name of the current Git branch:
$ python3 docker-build/scripts/builder.py build yourapp
$ docker run --rm yourapp:yourbranch-runtimeIf it blows up, you can run the image using a shell as the
entrypoint. You can then debug the problem in-place, e.g. try running
the custom entrypoint script directly, install new packages with
pip (so long as they donât require a compiler), and in
general play around until youâve figured out the issue.
If youâre running off of a Git repository, replace
yourbranch below with the name of the current Git
branch.
$ docker run -it yourapp:yourbranch-runtime bash
appuser@25444adb5dc0:~$ which python
/usr/local/bin/python
appuser@25444adb5dc0:~$ bash /home/appuser/docker-build/entrypoint.sh
...Notice that the applicationâs virtual environment is enabled by default, you donât need to change anything.
At this point you should be building images that work, so you can merge your temporary branch back into your main branch with a pull request or the equivalent.
Now that you have a working image, you probably want to set it up to build automatically whenever someone pushes a change to your version control repository.
At this point you have the image building locally, and so the next step is to make sure you have the credentials you need to push to an image registry, a server that stores images for you.
docker login. Once you have those credentials,
run docker login appropriately on your local machine.Letâs assume youâre using Quay, in which case the name will be
something like quay.io/yourorg/yourapp. If your chosen name
is already in use, test with a different image name, so you donât break
your production images!
Next we run two commands, a build and a push, giving the image name:
$ export IMAGE_NAME=quay.io/yourorg/yourapp # <-- change appropriately
$ python3 docker-build/scripts/builder.py build $IMAGE_NAME
$ python3 docker-build/scripts/builder.py push $IMAGE_NAMEYou can also push manually instead of using
builder.py push, just make sure you push both images. We
want both images pushed so that when you automate builds (see below) you
can get fast rebuilds. If youâre running off of a Git repository,
replace yourbranch below with the name of the current Git
branch:
$ docker image push $IMAGE_NAME:yourbranch-build
$ docker image push $IMAGE_NAME:yourbranch-runtimeYou should now see two images listed in the UI for your registry of
choice; in this case
quay.io/yourorg/yourapp:yourbranch-runtime and
quay.io/yourorg/yourapp:yourbranch-build. You can change
runtime and build to some other tag by editing
docker-build/build_settings.py appropriately.
You should also be able to pull the image:
$ docker pull $IMAGE_NAME:yourbranch-runtimeIf this worked, the next step is making the above run in your CI/build system.
There are a number of situations where you likely want Docker images to be built automatically:
main branch or
equivalent.See this article for a detailed explanation of why you want the latter.
Copy .github/workflows/docker.yml into your own
repositoryâs .github/workflows/ directory.
By default this uses GitHubâs Container Registry. You can change that
to any other container registry by editing the fields for the
docker/login-action: the username,
password, and registry fields.
You will also need to customize the environment variable called
BUILD_IMAGE_NAME in the docker.yml to match
complete image name youâre building, including the registry,
e.g. ghcr.io/yourorg/yourimage.
By default the main/master branch and
pull-requests to that branch have images built, as well as tags; if you
donât want that, youâll need to edit docker.yml
appropriately.
Copy the configuration from .gitlab-ci.yml into your own
configuration.
By default GitLabâs built-in Docker image registry is used, but if you want to push to another registry you can change the configuration appropriately.
IMPORTANT: The included configuration will reuse cached layers indefinitely, which means you will not get security updates (see this article). To fix that, you will need to manually set up a weekly build that rebuilds without caching:
EXTRA_BUILD_ARGS should be set to
--no-cache.For instructions on setting up scheduled pipelines see the documentation.
By default the main/master branch and
pull-requests to that branch have images built, as well as tags; if you
donât want that, youâll need to edit .gitlab-ci.yml
appropriately.
If you have some other build or CI system, you will need to run a build and push script manually in your CI system.
For example, if your image is quay.io/yourorg/yourapp,
you want a build script that looks something like this:
#!/bin/bash
set -euo pipefail # bash strict mode
docker login -u $REGISTRY_USER -p $REGISTRY_PASSWORD quay.io
export BUILD_IMAGE_NAME=quay.io/yourorg/yourapp
python3 docker-build/scripts/builder.py build $BUILD_IMAGE_NAME
python3 docker-build/scripts/builder.py push $BUILD_IMAGE_NAMETo rebuild images from scratch without caching (which you should do
weekly or daily to get security updates) you can run
docker-build/scripts/builder.py build --no-cache yourimagename.
Create a pull request to your main branch, and see if a Docker image gets built and pushed automatically.
Additional configuration will allow you to make sure you donât push broken images, and that your images donât include unnecessary files.
Before new images get pushed to the registry, itâs useful to run minimal test to ensure the new image is not completely broken. For example, a webserver can be tested by a sending a simple HTTP query to a running container.
Itâs worth while implementing such a test, and then adding it to your CI config right before the command that pushes to the registry. For more details see this article on Docker image smoke tests.
.dockerignore and COPY so you donât
package unnecessary files.dockerignore: Lists files that wonât get copied into
the Docker image.Dockerfile: The COPY . . line..dockerignore lists files that shouldnât be copied into
the Docker image. If you have any large data files in the repository, or
secrets, or any other files that shouldnât be copied into the Docker
image, add them here.
The file format is documented here.
Additionally, you can edit the Dockerfile so instead of
just copying everything in the current directory (as filtered through
.dockerignore) it only copies the files you need. For
example, if you only need the yourpackage/ and
data/ directories, you can change the
âCOPY . .â line to
âCOPY yourpackage/ data/ ./.
dive is
a tool for figuring out whatâs in your image, and where itâs coming
from.
docker-show-context
is another useful utility that lists which large files made it into the
Docker context. You can use it to figure out if there are any large
files being copied in.
The build will check the size of the runtime image, and complain if
itâs too large. This will help catch unexpectedly large images. To
change the configured maximum, edit MAX_IMAGE_SIZE_MB in
docker-build/build_settings.py.
If youâre doing local development, you might want to build the image
locally and have it install development dependencies like
black or flake8. You can do so by using the
--dev option; for example, this will build a Docker image
called yourapp with development dependencies installed:
$ ./docker-build/scripts/builder.py build --dev yourappThe template supports this in three ways:
requirements.txt to install
dependencies, you can provide a dev-requirements.txt.If youâre building manually, you can do:
$ docker build --build-arg INSTALL_DEV_DEPENDENCIES=1 .And if youâre using Docker Compose, your docker-compose.yaml can do:
version: "3.9"
services:
main:
context: .
args:
INSTALL_DEV_DEPENDENCIES: "1"By default images will be labeled with the current Git branch or tag, and the current git commit. You can see this metadata by running:
$ docker image inspect yourimage:yourtag | grep gitIf you want to add more labels, add them to
EXTRA_CLI_BUILD_ARGS in
docker-build/build_settings.py.
By default, the Docker images are tagged based on the current Git
branch or tag. If youâd like to customize this behavior, youâll want to
modify the various relevant options in
docker-build/build_settings.py.
docker buildIf you want to add additional arguments to docker build
(see the CLI
docs), you can add them via the EXTRA_CLI_BUILD_ARGS
list in build_settings.py. For example, if you want to run
docker build with
--secret id=MYSECRET,src=secret.txt you can do:
EXTRA_CLI_BUILD_ARGS.extend([
'--secret',
'id=MYSECRET,src=secret.txt',
])The Dockerfile should work out of the box with Compose,
with the caveat that if you use EXTRA_CLI_BUILD_ARGS in the
build_settings.py config, you will need to add those
arguments to the docker-compose.yml as well.
If you want to have Compose build the image, you can have the
following docker-compose.yml:
services:
main:
build: .Note that docker compose up doesnât automatically
rebuild the image when the code changes; youâll need to do
docker compose up --build.
Also note that you may get faster builds on Linux if you enable BuildKit (on macOS and Windows Buildkit is enabled by default):
$ export DOCKER_BUILDKIT=1By default the template uses BuildKit, which results in faster builds and supports additional features like build secrets.
Every application really requires two different sets of dependency descriptions:
The first set of dependencies can be used to easily update the second set of dependencies when you want to upgrade (e.g. to get security updates).
The second set of dependencies is what you should use to build the application, in order to get reproducible builds: that is, to ensure each build will have the exact same dependencies installed as the previous build.
Some tools that do this are pipenv and
poetry, but the easiest way to do that is with
uv.
Because of the use of caching, system packages wonât get security updates by default. This is why the default CI configuration above makes sure to rebuild the image from scratch, without any caching, once a week. Note that on GitLab CI this requires some manual setup.
Make sure you have set this up, otherwise you will eventually end up with insecure images.
You will then want to deploy these updates images to your production environment.
If you are using pinned dependencies, you will need an ongoing process to re-pin to new versions in order to get security updates and critical bugfixes.
GitHub can automatically notify you of security updates and can also update your dependencies in general.
There are also third-party services like Safety CLI to scan your dependencies for vulnerabilities.
You can and should define health checks for a Docker imageâa way for whatever system is running the container to check if the application is functioning correctly. The Docker image format itself supports defining health checks, however some systems like Kubernetes ignore these and have their own way of specifying these.
So check the documentation for the systems where you run your images, and add health checks.
The template is licensed using the attached
license. Essentially, you canât distribute the template code to any
other organization, and you may only be able to package a limited number
of services depending on your purchase. The only exceptions are the
entrypoint.sh and activate.sh, which you can
distribute as you wish (the latter is based on open source code, see the
file for details).
What you can do:
If you have any questions or problems please email me.
uv where possible for faster build times.uv.lock files.RUN_IN_PLACE is now off by default; previously it was on by
default.package-mode = false when using
Poetry.Dockerfile.PYTHON_VERSION to
BUILD_PYTHON_VERSION.When upgrading, the change are significant enough that you probably want to copy all the files over and just start from scratch.
Dockerfile.Dockerfile.To upgrade, copy in the updated
docker-build/scripts/install-dependencies.py script into
your repository.
poetry instead of pip, which makes supporting
third-party package repositories easier.To upgrade, copy in the updated
docker-build/scripts/install-dependencies.py script into
your repository.
Compared to version 1.x, the configuration has been simplified and is more template-like. In addition:
Upgrading from 1.0 is probably easiest by just by starting from scratch.If the 1.0 template is working for your application, there is however no pressing need to upgrade.