When to switch to Python 3.8
Python 3.8 was released in mid-October, but if you look at my recommendation for a base image for Docker it still talks about Python 3.7. And in fact, switching to Python 3.8 immediately can cause you problems.
Wondering when to switch your application? Here’s a quick rundown of the problems you’ll likely to encounter when switching major Python versions, and when it’s time to make the switch.
The problems with a new major Python release
1. Missing packages
opencv-python package has had something like 50 million downloads.
But as I write this, on October 28, 2019, if you try to install it on Python 3.8 it doesn’t work:
$ docker run python:3.8-slim pip install opencv-python ERROR: Could not find a version that satisfies the requirement opencv-python (from versions: none) ERROR: No matching distribution found for opencv-python
What’s going on? As of October 28, 2019, if you visit the files page for opencv-python on PyPI you’ll see that there is no Python 3.8 package available.
If you’re reading this at some later point, this will likely have been fixed. But in the immediate time after a new major release, you won’t be able to download wheels for many libraries.
Sometimes it’s just a matter of rebuilding a wheel. Sometimes some tweaking of code is needed. Whatever the issue, there is some lag between a major Python release and package availability.
2. Bugs in Python
Python 3.7.0 was released on June 27, 2018. 3.7.1 was released on Oct. 20, 2018, 4 months later, with a long list of bugfixes.
Of course, there’s always another bug fix release in the future, but given an 18-month window between major releases, lots of new code will get written but not extensively used. Which means a follow-up release with plenty of bug fixes.
3. You can’t use the syntax yet
New versions of Python often have new syntax, and that is the case with Python 3.8. However, other tools need to support the new syntax too—autoformatters, linters, and so on. Until they do, you can’t use the new syntax.
Why you should switch
Given that it takes work to upgrade—some additional testing, some tweaks to your code—it can be tempting to put off switching indefinitely. Why worry about incompatibilities, new versions, and what not, when you can just stick with your current version indefinitely?
The problem is that Python isn’t supported indefinitely, nor do libraries support all Python versions indefinitely. So if you’re running on a 5-year-old version of Python, switching becomes a Big Deal—you’ll often end up dealing with more significant cross-version changes in both Python and in libraries at the same time. Which makes upgrading scary.
Instead of one massive scary upgrade event every few years, it’s much safer to have a continuous, ongoing process of smaller upgrades. Whenever a new major Python version comes out, or a new major library version, wait a bit, and then switch.
When should you switch to a new major version of Python?
At a minimum, you will need to wait until:
- All your libraries explicitly support the new Python release.
- All the tools you rely on explicitly support the new Python release.
Wait a couple months after a new major Python release, try it out, and if dependencies are still missing keep trying again every month. If you’re particularly paranoid you can wait until the first point release, 3.8.1 in this case.
Learn how to build fast, production-ready Docker images—read the rest of the Docker packaging guide for Python.
Build production-ready Docker images—fast!
You want fast builds, small and secure images, operational correctness. And doing it all yourself will take you a week or more of effort.
Want to ship with confidence—in just hours?
Learn the faster way to build production-ready Python containers.
You need to get your job done, so how do you find time to learn new skills?
There’s not always time to learn new tools and technologies at work—but you still need to keep your skills sharp. And with so many tools and technologies to learn, you’re not even sure where to start.
Learn relevant, practical tools and techniques, quickly and efficiently, by signing up for my newsletter.
You’ll join over 1000 Python developers and data scientists getting weekly emails about software engineering best practices, from Docker packaging, to faster code, to better testing.