When should you upgrade to Python 3.12?
Python 3.12 is out now–but should you switch to it immediately? And if you shouldn’t upgrade just yet, when should you?
Immediately after the release in October 2023, you probably didn’t want to upgrade just yet. But from December 2023 and onwards, and certainly now, upgrading is definitely worth trying, though it may not succeed. To understand why, we need to consider Python packaging, the software development process, and take a look at the history of past releases.
The problems with a new major Python release
As with many open source projects, Python, Python libraries, and most of the toolchain and packages mentioned in this article are based on volunteer labor. None of this should be read as a complaint towards the people doing the maintenance, they’re doing hugely valuable work, released for free and often done for free, and everything takes time. In addition, because upgrades involve so many different groups, coordination and releases take even more time.
With that in mind, let’s consider the problems with using 3.12 immediately after a release, and in the following months:
1. Incompatible packages
In some cases, the code in a library is incompatible with a new Python release.
The Python 3.12 support for the Numba project, for example, is almost there, but not quite out as of Jan 17, 2024. There’s a release candidate though, so hopefully by the time you read this the final version will be out. As another example, as of Jan 17, 2024 PyTorch support for 3.12 is also a work in progress, with a prerelease out but still no stable release. For my own projects, I still have a couple that need updating.
Even if the code is compatible, if dependencies aren’t compatible that can still delay a release. For example, the initial blocker for doing a 3.12 release for my Rust-based projects was PyO3 doing a new release. Once they did that, I was able to work through my backlog.
Given many Python packages are maintained by volunteers with limited free time and working independently, it just takes time for updates to percolate through the system.
2. Missing binary packages
Even if the code is compatible, not all packages can be easily installed on 3.12 yet.
Typically, Python package maintainers upload compiled versions of their packages–known as “wheels”–to PyPI. When you install a package, you can just download the binary wheel and don’t need to compile it (unless you’re using Alpine Linux, in which case wheels may or may not be available).
But at least for a short time after 3.12’s release, some packages won’t have wheels for Python 3.12 uploaded yet. This is less of a problem these days compared to older Python releases, as:
- Package maintainers are putting in more and more effort to shorten this window, often releasing updated packages in advance of the Python final release.
- Some packages release binary wheels using the forward-compatible “abi3” format, which means newer Python releases just work with no extra effort.
Nonetheless, there will be packages that won’t have binary wheels yet. Best case, that means installation will take a lot longer as you have to compile the code on installation. Worst case, compilation will fail because you don’t have relevant compile-time dependencies installed.
Important: Make sure you upgrade pip before using it. Older versions might not support the binary wheels for 3.11 on Linux, and instead
pipwill try to build from source.
Conda support
If you’re using Conda-Forge, the Python 3.12 rollout happens pretty quickly. A lot of it is automated, one of the benefits of Conda-Forge over PyPI, so as long as the underlying packages already support 3.12, it happens pretty quickly across the whole package repository.
3. Bugs in Python
Python 3.12.0 was released in October 2023. 3.12.1 was released in December 2023, with more than 400 bugfixes.
Of course, there’s always another bug fix release in the future, but given a 12-month window between major releases, lots of new code will get written but not extensively used. Which means a follow-up release with plenty of bug fixes.
Things are getting better!
I’ve been updating this article over the years since 3.9, and the situation has gotten significantly better since then. Every release I’ve reduced the caveats, and projects have sped up their iteration cycles so they have support for newer releases even faster.
Is upgrading worth it?
Given that it takes work to upgrade—some additional testing, some tweaks to your code—it can be tempting to put off upgrading Python versions indefinitely. Why worry about incompatibilities, new versions, and what not, when you can just stick with your current version indefinitely?
The problem is that Python isn’t supported indefinitely, nor do libraries support all Python versions indefinitely; eventually you’ll stop getting security updates and bug fixes. Python 3.7 ended security updates in June 2023. 3.8 will be next, dropping security updates in October 2024.
So sooner or later you will have to upgrade. And if you’re running on a 5-year-old version of Python, switching becomes a Big Deal—you’ll often end up dealing with more significant cross-version changes in both Python and in libraries at the same time. Which makes upgrading scary.
Instead of one massive scary upgrade event every few years, it’s much safer to have a continuous, ongoing process of smaller upgrades. Whenever a new major Python version comes out, or a new major library version, wait a bit, and then switch.
The week Python 3.12 was released, you probably didn’t want to upgrade, but you certainly wanted to be using 3.11. If you’re far enough behind, do a series of upgrades: 3.9, 3.10, 3.11, and finally 3.12.
When should you switch your application to Python 3.12?
At a minimum, after a major new release you will need to wait until:
- All your libraries explicitly support the new Python release.
- All the tools you rely on explicitly support the new Python release.
Now that December 2023 has passed, 3.12.1 is out, and many more libraries have updated, it’s worth trying to upgrade. If the dependencies you require are still missing 3.12 support, keep trying again every month or two.
