When to switch to Python 3.9

Python 3.9 is planned for release in October 2020. But should you switch to it immediately?

To answer that question, we can consider the switchover to Python 3.8. Python 3.8 was released in mid-October 2019, but if you were looking at my recommendation for a base image for Docker it talked about Python 3.7 until late January 2020. And in fact, switching to 3.8 immediately—or for that matter any major Python release, like 3.9 when it comes out—can cause you problems.

Wondering when to switch your application? Here’s a quick rundown of the problems you’ll likely to encounter when switching major Python versions, and when it’s time to make the switch.

The problems with a new major Python release

1. Missing packages

The opencv-python package has had something like 50 million downloads. It’s popular!

But when I first wrote this article, on October 28, 2019, if you tried to install it on Python 3.8 it didn’t work:

$ docker run python:3.8-slim pip install opencv-python
ERROR: Could not find a version that satisfies the requirement opencv-python (from versions: none)
ERROR: No matching distribution found for opencv-python

What was going on? If you visit the files page for opencv-python on PyPI you’ll see that that the Python 3.8 package was only added in late November, more than a month after the release of Python 3.8.

The problem is that in the immediate time frame after a new major release, you won’t be able to download wheels for many libraries.

Sometimes it’s just a matter of rebuilding a wheel. Sometimes some tweaking of code is needed. Whatever the issue, there is some lag between a major Python release and package availability.

2. Bugs in Python

Python 3.7.0 was released on June 27, 2018. 3.7.1 was released on Oct. 20, 2018, 4 months later, with a long list of bugfixes.

Of course, there’s always another bug fix release in the future, but given an 18-month window between major releases, lots of new code will get written but not extensively used. Which means a follow-up release with plenty of bug fixes.

3. You can’t use the syntax yet

New versions of Python often have new syntax, and that is the case with Python 3.8. However, other tools need to support the new syntax too—autoformatters, linters, and so on. Until they do, you can’t use the new syntax.

Why you should switch

Given that it takes work to upgrade—some additional testing, some tweaks to your code—it can be tempting to put off switching indefinitely. Why worry about incompatibilities, new versions, and what not, when you can just stick with your current version indefinitely?

The problem is that Python isn’t supported indefinitely, nor do libraries support all Python versions indefinitely. So if you’re running on a 5-year-old version of Python, switching becomes a Big Deal—you’ll often end up dealing with more significant cross-version changes in both Python and in libraries at the same time. Which makes upgrading scary.

Instead of one massive scary upgrade event every few years, it’s much safer to have a continuous, ongoing process of smaller upgrades. Whenever a new major Python version comes out, or a new major library version, wait a bit, and then switch.

When should you switch to a new major version of Python?

At a minimum, you will need to wait until:

  1. All your libraries explicitly support the new Python release.
  2. All the tools you rely on explicitly support the new Python release.

Wait a couple months after a new major Python release, try it out, and if dependencies are still missing keep trying again every month. If you’re particularly paranoid you can wait until the first point release, 3.8.1 in this case.

These days you can should be using 3.8—but when 3.9 comes out, the whole cycle will begin again. Personally I wouldn’t switch to Python 3.9 until at least two months after it’s released, January 2021 or so: this gives you a minor release for bugfixes, and some time for the toolchain to update, and for libraries to put out 3.9-compatible packages.

Learn how to build fast, production-ready Docker images—read the rest of the Docker packaging guide for Python.