When to switch to Python 3.8
Python 3.8 was released in mid-October 2019, but if you were looking at my recommendation for a base image for Docker it talked about Python 3.7 until late January 2020. And in fact, switching to 3.8 immediately—or for that matter any major Python release—can cause you problems.
Wondering when to switch your application? Here’s a quick rundown of the problems you’ll likely to encounter when switching major Python versions, and when it’s time to make the switch.
The problems with a new major Python release
1. Missing packages
opencv-python package has had something like 50 million downloads.
But when I first wrote this article, on October 28, 2019, if you tried to install it on Python 3.8 it didn’t work:
$ docker run python:3.8-slim pip install opencv-python ERROR: Could not find a version that satisfies the requirement opencv-python (from versions: none) ERROR: No matching distribution found for opencv-python
What was going on? If you visit the files page for opencv-python on PyPI you’ll see that that the Python 3.8 package was only added in late November, more than a month after the release of Python 3.8.
The problem is that in the immediate time frame after a new major release, you won’t be able to download wheels for many libraries.
Sometimes it’s just a matter of rebuilding a wheel. Sometimes some tweaking of code is needed. Whatever the issue, there is some lag between a major Python release and package availability.
2. Bugs in Python
Python 3.7.0 was released on June 27, 2018. 3.7.1 was released on Oct. 20, 2018, 4 months later, with a long list of bugfixes.
Of course, there’s always another bug fix release in the future, but given an 18-month window between major releases, lots of new code will get written but not extensively used. Which means a follow-up release with plenty of bug fixes.
3. You can’t use the syntax yet
New versions of Python often have new syntax, and that is the case with Python 3.8. However, other tools need to support the new syntax too—autoformatters, linters, and so on. Until they do, you can’t use the new syntax.
Why you should switch
Given that it takes work to upgrade—some additional testing, some tweaks to your code—it can be tempting to put off switching indefinitely. Why worry about incompatibilities, new versions, and what not, when you can just stick with your current version indefinitely?
The problem is that Python isn’t supported indefinitely, nor do libraries support all Python versions indefinitely. So if you’re running on a 5-year-old version of Python, switching becomes a Big Deal—you’ll often end up dealing with more significant cross-version changes in both Python and in libraries at the same time. Which makes upgrading scary.
Instead of one massive scary upgrade event every few years, it’s much safer to have a continuous, ongoing process of smaller upgrades. Whenever a new major Python version comes out, or a new major library version, wait a bit, and then switch.
When should you switch to a new major version of Python?
At a minimum, you will need to wait until:
- All your libraries explicitly support the new Python release.
- All the tools you rely on explicitly support the new Python release.
Wait a couple months after a new major Python release, try it out, and if dependencies are still missing keep trying again every month. If you’re particularly paranoid you can wait until the first point release, 3.8.1 in this case.
As of January 2020, it’s reasonable to switch to 3.8—but when 3.9 comes out, the whole cycle will begin again.
Learn how to build fast, production-ready Docker images—read the rest of the Docker packaging guide for Python.
You’re about to ship your Python application into production using Docker: your images are going to be critical infrastructure. You can’t afford slow builds or security breaches.
But you also can’t afford to waste a week doing research. Developer time is expensive—save money by using the Python on Docker packaging checklist.
Learn practical Docker and Python software engineering skills, every week
You need to stay competitive in the job market—but there's too much to learn, and you don’t know where to start.
Sign up for my newsletter, and join over 1600 Python developers and data scientists learning practical tools and techniques, from Docker packaging to Python best practices, with a free new article in your inbox every week.