10+ technologies, 60+ best practices, and you need to figure it out all on your own

You’re packaging your Python application for production with Docker—and production use means you need to implement best practices: for security, speed, reproducibility, debuggability.

Unfortunately, to do that you’re going to have to use and understand a whole pile of tools, technologies, and techniques, and apply them correctly too:

  • Shell scripting,
  • the Dockerfile format,
  • Docker’s build caching,
  • signal handling,
  • Python packaging (which might involve setup.py, requirements.txt, Poetry, Pipenv, or Conda),
  • Git,
  • CI,
  • security updates (for system packages and Python dependencies),
  • and on and on and on.

All of these technologies interact with each other, sometimes in unexpected ways. For example, Docker’s build caching can prevent your system packages from getting security updates.

This is ops: there’s a lot of details. And you can’t abstract the details away, the details are the point.

So how exactly are you going to implement these best practices?

  1. You’re going to have figure out what they are—I’ve come up with over 60, and my list keeps growing.
  2. Then, you’re going to have to figure out where to get started, what to do next, what’s required, and what’s optional.

Doing all this on your own is overwhelming, not to mention a massive time sink.

Docker images, built right and built well

Here’s how you can build your Docker image the right way, without getting overwhelmed.

First, you need to start with the best practices that really matter. That means making sure your code is packaged with up-to-date, secure dependencies, and that a rebuild today gives you the same results as a build last month.

Then you ensure your service is running smoothly and reliably. And you prepare in advance so that if your application ever does crash, you’ll be able to easily discover what happened and why.

And finally, you focus on fast builds and small images, to ensure a productive development process and efficient artifacts.

A packaging process that will take you from zero to production

And this is where the Python on Docker Production Quickstart comes in.

The Quickstart guide will:

  • Teach you a clear, step-by-step iterative process to packaging your application with Docker, starting with the most important parts like security, and ending with optimizations like build time and image size.
  • Recommend specific best practices for each step in the process.
  • Provide a reference covering over 60 best practices, with examples and links to further reading. This includes 25 best practices you won’t find elsewhere on this site.
  • Respect your limited time. A concise summary, the Quickstart includes ~80 pages of to-the-point information.

“I’ve been using [it] a lot over the last two weeks and it’s been absolutely fantastic.

Learning about multi-stage builds has meant I can combine a Django/React app into one Dockerfile. This also lead to me discovering compose build sections can target a named stage so now I’ve got local dev using the same Dockerfile, which is great.”

– George Hickman

The included best practices cover the following areas:

  • Security.
  • Running in CI.
  • Easier debugging.
  • Operational correctness.
  • Reproducible builds.
  • Faster builds.
  • Smaller images, including multi-stage builds.
  • Application-specific best practices.
  • Conda.
  • Poetry.

“We applied the secrets handling and multi-stage builds from the Quickstart. Having said that, everything in it was an eye opener.”

— Gregory Ronin, Software Enginer at Deloitte Digital

Get ready for production, today


Bundle #1: Python on Docker Production Quickstart ($79)

It’s super valuable. I now have a big list of TODOs to update our build scripts and Dockerfiles.”

– Eric Pederson

Includes the Python on Docker Production Quickstart guide:

  • Includes PDF guide, as well as HTML version for easier copy/pasting of example code.
  • Focuses specifically on Python running in production.
  • Provides a step-by-step iterative implementation plan.
  • Covers over 60 best practices.
  • All updates to the 1st edition, delivered by email (last updated Nov 13, 2020).
  • 100% money-back guarantee.
Buy the Quickstart for US$79

Note: I prefer not to take money from teams working on projects for the military, prisons, police, fossil fuel extraction, surveillance, national security, or the like. If that applies to you, please don’t purchase this product.


Bundle #2: Ship faster ($119)

Includes:

  1. The Python on Docker Production Quickstart.
  2. A simple template implementing 17 different best practices for you, so you can get started even faster.
    • Save a few hours work: the template implements the common best practices almost everyone needs.
    • Learn the best practices in more details: includes a screencast video explaining how the template is implemented.
    • Use it anywhere: An open source license, so you can use the template anywhere.
Buy the Quickstart + Simple Template for US$119

Note: I prefer not to take money from teams working on projects for the military, prisons, police, fossil fuel extraction, surveillance, national security, or the like. If that applies to you, please don’t purchase this product.


Bundle #3: Package your application in just 3 hours ($329)

What users are saying:

“The [production-ready] template worked great 🙂

The app using it is now live within the hospital’s ICU unit, and I’ll be using the same template to deploy an API (still in development) on the same infrastructure.”

– Nel Swanepol, University College London

Includes:

  1. The Python on Docker Production Quickstart.
  2. The simple template.
  3. The Production-Ready Python Containers template.

Implementing far more than the simple template, the Production-Ready Python Containers template lets you quickly ship to production. The first time you go through the onboarding tutorial, I would expect a simple WSGI application to take as little as 30 minutes, and I’ve seen a new user get a complex application going in just 2-3 hours.

With practice, packaging gets even faster. For an admittedly simple case, a WSGI app with no special system packages, I was able to package a Python application in just 3 minutes.




The Production-Ready Python Containers template:

  • Includes sample configurations for GitHub Actions, GitLab CI, and CircleCI.
  • Supports pipenv and poetry, as well as requirements.txt, setup.py, and both run-from-check and run-from-install.
  • Supports WSGI applications with an optional gunicorn configuration.
  • If you’re compiling extension modules, get both fast builds and small images using multi-stage builds.
  • Lets you access private repos and package repositories using secure build secrets.
  • Supports Ubuntu, Debian, CentOS, and RHEL out of the box.
Buy the Quickstart + Simple Template + Production-Ready Template for US$329

Note: I prefer not to take money from teams working on projects for the military, prisons, police, fossil fuel extraction, surveillance, national security, or the like. If that applies to you, please don’t purchase this product.

Quickstart Changelog

As mentioned above, all of the purchase options will includes updates to the 1st edition of the Quickstart; here are the updates that have been made so far.

November 13, 2020

  • Documented PEP 517 Poetry installation usage.
  • A large number of minor code fixes throughout, as well as some typos in the text.
  • HTML version is now included, for easier copy/pasting.

October 22, 2020

  • Added three best practices for Poetry.
  • Added best practice on adding .git to .dockerignore, and what do when you can’t.

October 14, 2020

Added three best practices for smaller Conda-based images.

June 17, 2020

More best practices:

  • Various BuildKit features that help speed up builds.
  • Dropping capabilities.
  • Avoiding listening on ports < 1024.
  • Running a different command altogether based on command-line arguments.
  • Bytecode compilation.
  • Additional image size optimization in multi-stage builds.
  • Warm up the build cache for per-branch builds.
  • Requirements for running on Heroku and Google Cloud Run.
  • BuildKit ssh-agent forwarding.
  • BuildKit secrets when using Docker Compose.

Other tweaks and improvements throughout the text.

June 8, 2020

The Checklist has been renamed, and is now known as a Quickstart. To help make that change:

  • Added a new introductory chapter with a plan to help you figure out which best practices to implement when.
  • Tweaked the chapter structure.

Additionally:

  • Added a best practice on finding large layers.
  • Split off init into its own best practice.
  • Explained the goal of responding to health checks quickly more broadly, rather than in specific implementation terms of process/thread pool.
  • Added more nuance to the section on updating dependencies once a month.
  • Restored the Pipenv instructions, since it’s now being maintained again.
  • Make the DEBIAN_FRONTEND best practice standalone.

June 2, 2020

Added multiple new best practices:

  • Additional tips on improving build caching.
  • A better default tag.
  • For better reproducibility, you can create a custom base image.
  • Size checks for images.
  • Security scanners.

Also updated existing best practices:

  • You can make a build arg available at runtime by using ENV.
  • For build secrets another alternative is short term keys.

April 27, 2020

  • Added new best practice on timely security updates via automatic notifications.
  • Switched some examples from shell session transcripts to Dockerfile or shell script.
  • Noted docker build support for targeting named stages earlier in the checklist.

April 1, 2020

  • Documented two-stage install with Poetry 1.0+.
  • Added link to docker-autoheal.

February 24, 2020

Added many more examples:

  • Configuring logging.
  • A smoke test.
  • Passing in secrets with BuildKit.
  • .dockerignore file.
  • System package upgrade script for CentOS/RHEL.
  • Dockerfile healthcheck.
  • And a few more expanded examples here and there.

About me

Hi, I’m Itamar Turner-Trauring.

I’ve been writing Python since 1999, and I first started using Docker in 2014, when I was part of a team that wrote one of the first distributed storage backends for Docker.

I’ve since built Telepresence, a remote development tool for Kubernetes that was adopted as a Cloud Native Computing Foundation sandbox project and has more than 2000 stars on GitHub. I’ve also deployed a number of production Python applications as Docker images.

Over the past year I’ve been researching Docker packaging for Python in production, resulting in a free guide elsewhere on this site, live training classes, an introductory book, a template implementing these best practices, and of course this quickstart.