10+ technologies, 70+ best practices, and you need to figure it out all on your own

You’re packaging your Python application for production with Docker—and production use means you need to implement best practices: for security, speed, reproducibility, debuggability. And of course you want fast builds and small images.

Unfortunately, to do that you’re going to have to use and understand 50 years of accumulated tools, technologies, and techniques:

  • Shell scripting,
  • the Dockerfile format,
  • Docker’s build caching,
  • signal handling,
  • Python packaging (which might involve setup.py, requirements.txt, Poetry, Pipenv, or Conda),
  • Git,
  • CI,
  • security updates (for system packages and Python dependencies),
  • and on and on and on.

All of these technologies interact with each other, sometimes in unexpected ways. For example, Docker’s build caching can prevent your system packages from getting security updates.

This is ops: there’s a lot of details. And you can’t abstract the details away, the details are the point.

So how exactly are you going to implement these best practices?

  1. You’re going to have figure out what they are—I’ve come up with over 70, and my list keeps growing.
  2. Then, you’re going to have to figure out where to get started, what to do next, what’s required, and what’s optional.

Doing all this on your own is overwhelming, not to mention a massive time sink.

Docker images, built right and built well

Here’s how you can build your Docker image the right way, without getting overwhelmed.

First, you need to start with the best practices that really matter. That means making sure your code is packaged with up-to-date, secure dependencies, and that a rebuild today gives you the same results as a build last month.

Then you ensure your service is running smoothly and reliably. And you prepare in advance so that if your application ever does crash, you’ll be able to easily discover what happened and why.

And finally, you focus on fast builds and small images, to ensure a productive development process and efficient artifacts.

A packaging process that will take you from zero to production

And this is where the Python on Docker Production Handbook comes in.

The Handbook will:

  • Teach you a clear, step-by-step iterative process to packaging your application with Docker, starting with the most important parts like security, and ending with optimizations like build time and image size.
  • Recommend specific best practices for each step in the process.
  • Provide a reference covering over 70 best practices, with examples and links to further reading. This includes 30 best practices you won’t find elsewhere on this site.
  • Up-to-date with Docker 20.10 and BuildKit 1.2, with specific information on Poetry, Pipenv, and Conda packaging.
  • Respect your limited time. A concise summary, the Handbook includes ~110 pages of to-the-point information.

“I’ve found it very helpful, just having a general guide off best practices to anything to do with deployment. I find this an excellent complement to Two Scoops of Django.”

— Joe Selvik

“I’ve been using [it] a lot over the last two weeks and it’s been absolutely fantastic.

Learning about multi-stage builds has meant I can combine a Django/React app into one Dockerfile. This also lead to me discovering compose build sections can target a named stage so now I’ve got local dev using the same Dockerfile, which is great.”

– George Hickman

The included best practices cover the following areas:

  • Security.
  • Running in CI.
  • Easier debugging.
  • Operational correctness.
  • Reproducible builds.
  • Faster builds.
  • Smaller images, including multi-stage builds.
  • Application-specific best practices.
  • Conda.
  • Poetry.
  • Pipenv.

“We applied the secrets handling and multi-stage builds from the Handbook. Having said that, everything in it was an eye opener.”

— Gregory Ronin, Software Enginer at Deloitte Digital

Get ready for production, today


Bundle #1: Python on Docker Production Handbook ($79)

It’s super valuable. I now have a big list of TODOs to update our build scripts and Dockerfiles.”

– Eric Pederson

Includes the Python on Docker Production Handbook:

  • 110-page PDF, plus HTML version for easier copy/pasting of example code.
  • Focuses specifically on Python running in production.
  • Provides a step-by-step iterative implementation plan.
  • Covers over 70 best practices, updated for Docker 20.10 and BuildKit 1.2.
  • All updates to the 1st edition, delivered by email (last updated March, 2021).
  • 100% money-back guarantee.
Buy the Handbook for US$79

Note: I prefer not to take money from teams working on projects for the military, prisons, police, fossil fuel extraction, surveillance, national security, or the like. If that applies to you, please don’t purchase this product.


Bundle #2: Ship faster ($119)

Includes:

  1. The Python on Docker Production Handbook.
  2. A simple template implementing 17 different best practices for you, so you can get started even faster.
    • Save a few hours of work: the template implements the common best practices almost everyone needs.
    • Learn the best practices in more details: includes a screencast video explaining how the template is implemented.
    • Use it anywhere: An open source license, so you can use the template anywhere.
  3. 100% money-back guarantee.
Buy the Handbook + Simple Template for US$119

Note: I prefer not to take money from teams working on projects for the military, prisons, police, fossil fuel extraction, surveillance, national security, or the like. If that applies to you, please don’t purchase this product.


Bundle #3: Package your application in just 3 hours ($329)

What users are saying:

“The [production-ready] template worked great 🙂

The app using it is now live within the hospital’s ICU unit, and I’ll be using the same template to deploy an API (still in development) on the same infrastructure.”

– Nel Swanepol, University College London

Includes:

  1. The Python on Docker Production Handbook.
  2. The simple template.
  3. The Production-Ready Python Containers template.
  4. 100% money-back guarantee.

Implementing far more than the simple template, the Production-Ready Python Containers template lets you quickly ship to production. The first time you go through the onboarding tutorial, I would expect a simple WSGI application to take as little as 30 minutes, and I’ve seen a new user get a complex application going in just 2-3 hours.

With practice, packaging gets even faster. For an admittedly simple case, a WSGI app with no special system packages, I was able to package a Python application in just 3 minutes.




The Production-Ready Python Containers template:

  • Includes sample configurations for GitHub Actions, GitLab CI, and CircleCI.
  • Supports pipenv and poetry, as well as requirements.txt, setup.py, and both run-from-check and run-from-install.
  • Supports WSGI applications with an optional gunicorn configuration.
  • If you’re compiling extension modules, get both fast builds and small images using multi-stage builds.
  • Lets you access private repos and package repositories using secure build secrets.
  • Supports Ubuntu, Debian, CentOS, and RHEL out of the box.
Buy the Handbook + Simple Template + Production-Ready Template for US$329

Note: I prefer not to take money from teams working on projects for the military, prisons, police, fossil fuel extraction, surveillance, national security, or the like. If that applies to you, please don’t purchase this product.

Changelog

As mentioned above, all of the purchase options will includes updates to the 1st edition of the Handbook; here are the updates that have been made so far.

March 16, 2021

  • When using BuildKit you should always set --build-arg BUILDKIT_INLINE_CACHE=1 if you want --cache-from to work.
  • Noted that official Python Docker image is slower than Ubuntu.

February 8, 2021

  • The Quickstart has been renamed to the Handbook; at 100 pages, it’s getting more than just introductory.
  • New best practices:
    • Installing packages with pipenv without exporting.
    • Keeping pipenv separate from application code.
    • Recommend docker build --label over LABEL.
    • Caching package downloads across builds using BuildKit.
    • Avoiding dev dependency installs in Poetry.
    • Conda dependency locking using conda-lock.
  • Removed references to CentOS, as it is no longer a stable base image.
  • Added link to RedHat’s Docker base image.
  • Noted Podman can be used with BuildKit.
  • Noted need for --keep-outdated when using pipenv lock.
  • Documented how to make Docker Compose use BuildKit.
  • Noted need to pass --no-capture-output to conda run (thanks to Joe Selvik).

December 17, 2020

Updates for Docker 20.10 and a stable BuildKit.

  • Added new chapter covering different Docker releases and BuildKit.
  • Switched all BuildKit examples to use the new stable docker/dockerfile:1.2 version.
  • Documented getting BuildKit secrets from environment variables.

November 13, 2020

  • Documented PEP 517 Poetry installation usage.
  • A large number of minor code fixes throughout, as well as some typos in the text.
  • HTML version is now included, for easier copy/pasting.

October 22, 2020

  • Added three best practices for Poetry.
  • Added best practice on adding .git to .dockerignore, and what do when you can’t.

October 14, 2020

Added three best practices for smaller Conda-based images.

June 17, 2020

More best practices:

  • Various BuildKit features that help speed up builds.
  • Dropping capabilities.
  • Avoiding listening on ports < 1024.
  • Running a different command altogether based on command-line arguments.
  • Bytecode compilation.
  • Additional image size optimization in multi-stage builds.
  • Warm up the build cache for per-branch builds.
  • Requirements for running on Heroku and Google Cloud Run.
  • BuildKit ssh-agent forwarding.
  • BuildKit secrets when using Docker Compose.

Other tweaks and improvements throughout the text.

June 8, 2020

The Checklist has been renamed, and is now known as a Quickstart. To help make that change:

  • Added a new introductory chapter with a plan to help you figure out which best practices to implement when.
  • Tweaked the chapter structure.

Additionally:

  • Added a best practice on finding large layers.
  • Split off init into its own best practice.
  • Explained the goal of responding to health checks quickly more broadly, rather than in specific implementation terms of process/thread pool.
  • Added more nuance to the section on updating dependencies once a month.
  • Restored the Pipenv instructions, since it’s now being maintained again.
  • Make the DEBIAN_FRONTEND best practice standalone.

June 2, 2020

Added multiple new best practices:

  • Additional tips on improving build caching.
  • A better default tag.
  • For better reproducibility, you can create a custom base image.
  • Size checks for images.
  • Security scanners.

Also updated existing best practices:

  • You can make a build arg available at runtime by using ENV.
  • For build secrets another alternative is short term keys.

April 27, 2020

  • Added new best practice on timely security updates via automatic notifications.
  • Switched some examples from shell session transcripts to Dockerfile or shell script.
  • Noted docker build support for targeting named stages earlier in the checklist.

April 1, 2020

  • Documented two-stage install with Poetry 1.0+.
  • Added link to docker-autoheal.

February 24, 2020

Added many more examples:

  • Configuring logging.
  • A smoke test.
  • Passing in secrets with BuildKit.
  • .dockerignore file.
  • System package upgrade script for CentOS/RHEL.
  • Dockerfile healthcheck.
  • And a few more expanded examples here and there.

About me

Hi, I’m Itamar Turner-Trauring.

I’ve been writing Python since 1999, and I first started using Docker in 2014, when I was part of a team that wrote one of the first distributed storage backends for Docker.

I’ve since built Telepresence, a remote development tool for Kubernetes that was adopted as a Cloud Native Computing Foundation sandbox project and has more than 2000 stars on GitHub. I’ve also deployed a number of production Python applications as Docker images.

Over the past year I’ve been researching Docker packaging for Python in production, resulting in a free guide elsewhere on this site, live training classes, an introductory book, a template implementing these best practices, and of course this handbook.