“Externally managed environments”: when PEP 668 breaks pip
You’re on a new version of Linux, you try a
pip install, and it errors out, talking about “externally managed environments” and “PEP 668”.
What’s going on?
How do you solve this?
- What the problem looks like, and what causes it.
- The places you are likely to encounter it.
- A variety of solutions, depending on your use case.
Symptoms: failed installs
Consider the following Dockerfile, using a pre-release of Debian “Bookworm” 12, which will be released in June 2023:
FROM debian:bookworm RUN apt-get update && apt-get -y install python3 python3-pip RUN pip install flask
This seems pretty straightforward, similar to many Docker examples you’ll find. And yet when built, it fails:
> [3/3] RUN pip install flask: error: externally-managed-environment × This environment is externally managed To install Python packages system-wide, try apt install python3-xyz, where xyz is the package you are trying to install. If you wish to install a non-Debian-packaged Python package, create a virtual environment using python3 -m venv path/to/venv. Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make sure you have python3-full installed. If you wish to install a non-Debian packaged Python application, it may be easiest to use pipx install xyz, which will manage a virtual environment for you. Make sure you have pipx installed. See /usr/share/doc/python3.11/README.venv for more information. note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages. hint: See PEP 668 for the detailed specification.
Note: Outside any specific best practice being demonstrated, the Dockerfiles in this article are not examples of best practices, since the added complexity would obscure the main point of the article.
Need to ship quickly, and don’t have time to figure out every detail on your own? Read the concise, action-oriented Python on Docker Production Handbook.
Where you’ll encounter this
As we saw above, this problem occurs with the system Python from Debian 12. It’ll also happen in Ubuntu 23.04 or later. This is less relevant for Docker builds, since that’s not a long-term support release, but it will become an issue once Ubuntu 24.04 LTS is released.
And other Linux distributions will likely have this issue as well. For example, as of May 2023, Fedora has a proposal to turn this on.
Older Linux distributions, like Ubuntu 22.04, Debian 11, and current RHEL and clones, do not have this problem.
Additionally, if you use the “official” Python base images provided by Docker, you won’t have any issues:
FROM python:3.11-slim-bullseye RUN pip install flask
$ docker build . ... => => writing image sha256:483c6c3804fe2681acf61d79af326cd81c1d1824cd45f4b7c04bd4f7728 0.0s $
This image is based on Debian 11, but that’s not why it works.
I would expect future
python images based off of Debian 12 to build just fine too.
||Not available yet, but likely yes|
Why the difference? And what is PEP 668 and why does it break things?
System packages vs.
Linux distributions package a variety of software, both libraries and applications. Some of that software is written in Python.
For example, on Ubuntu 20.04:
update-managerpackage, the GUI for installing system updates, depends on
python3-yamlpackage is version 5.3.1.
- This is based on the
python3-yamlPyPI package, version 6.0 at the time of writing.
So if I start a Python prompt, I can import
$ docker run -it ubuntu:20.04 bash root@d67511efab81:/# apt-get update && apt-get install --no-install-recommends python3-pip update-manager ... root@d67511efab81:/# python3 Python 3.8.10 (default, Mar 13 2023, 10:26:41) [GCC 9.4.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import yaml >>> yaml.__version__ '5.3.1'
That’s bad if you want your code to run reproducibly: you now have a specific version of Python’s YAML library installed and importable that might not match the one your code expects.
And what if I then
pip install --upgrade pyyaml as root?
root@d67511efab81:/# pip install --upgrade pyyaml ... root@d67511efab81:/# python3 Python 3.8.10 (default, Mar 13 2023, 10:26:41) [GCC 9.4.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import yaml >>> yaml.__version__ '6.0'
update-manager tool still work with PyYAML 6 instead of the 5.3 it expected? Who knows!
To summarize, we have two core problems:
- Leakage: System-provided Python packages are unexpectedly importable.
pip-installed packages can break system packages.
Solving the mismatch
The first problem, that of leakage, is solvable with virtualenvs, which give you isolated environments where any packages installed by the operating system won’t affect you.
root@d67511efab81:/# python3 -m venv myvenv root@d67511efab81:/# myvenv/bin/python Python 3.8.10 (default, Mar 13 2023, 10:26:41) [GCC 9.4.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import yaml Traceback (most recent call last): File "<stdin>", line 1, in <module> ModuleNotFoundError: No module named 'yaml'
The second problem, of
pip install breaking system packages, is solved by PEP 668.
PEPs are Python Enhancement Proposals: proposals to improve the Python language or core tooling in some way.
PEP 668 provides a mechanism Linux distributions to tell
pip that it can’t install packages outside a virtualenv.
And the latest versions of Debian and Ubuntu have decided, quite reasonably, to opt-in to this mechanism.
You don’t want a stray
pip install breaking your operating system packages!
That’s the source of the “externally managed environment” error: you’re trying to
pip install into an environment managed by the Linux distribution.
pip install --user breaks too
Linux distributions that have opted-in to PEP 668 will also prevent
pip install --user, a mechanism that lets you install user-specific packages in your home directory.
While this mechanism is better than
pip install as root, since it definitely won’t overwrite files installed by the operating system, it still has the issue that the modules being imported might be different than the ones system packages expect.
For example, if you
pip install --user pyyaml, that might result in
update-manager importing a different version of PyYAML than it expected; this is leakage from the user to the system.
Solving the “externally managed environment” problem
There are a number of approaches you can use if you’ve encountered an “externally managed environment” error.
During development: virtualenvs
The most general solution is installing everything in a virtualenv: it’s isolated from the system Python so you can do whatever you want without worrying about leakage or breakage.
$ python -m venv ./myvenv $ . ./myvenv/bin/activate (venv)$ pip install flask
For Docker packaging: Don’t use the system Python
First, virtualenvs also work for Docker packaging, so you could just do that.
But if you don’t want to use a virtualenv, there’s another option. The motivation for PEP 668 was preventing conflicts between system packages and user packages. If you’re using a version of Python that was not installed by the operating system, then such conflicts are not an issue.
pythonDocker images: For the Docker use case, I expect that future
pythonbase images will continue to not have any restrictions on
pip install, because the Python they provide is not the operating system Python, it’s a separate install.
- Conda: Similarly, Conda installs its own Python.
For system-packaged applications: system package manager
If you want to install an application written in Python, you can just install the version provided by your Linux distribution, for example with
snap on Ubuntu.
For Python tools where you want the latest version: pipx
Consider tools like
mypy, where the fact they’re implemented in Python is more of an implementation detail.
Still, you often want the latest version, and your Linux distribution might be out of date.
One good solution is
pipx lets you install Python applications in virtualenvs, one per application, so they don’t break each other (or system packages!).
Because of the bootstrapping problem (how do you install
pip won’t work?) you’ll want to use the
pipx from your Linux distribution.
$ sudo apt-get install -y pipx ... $ pipx ensurepath
And now you can do:
$ pipx install black $ pipx install mypy
Packaging is a pain: be kind!
Linux distributions with a focus on stability have different use cases than developers writing code, who in turn have different use cases than developers packaging their code, or developers installing tools. Python sits in the middle, trying to satisfy all these use cases while still allowing your code to run.
While encountering these sort of changes can be frustrating, keep in mind that everyone involved likely spent a lot of time thinking through alternatives and consequences, and didn’t make these changes lightly. I have yet to see any packaging setup that isn’t painful, and that seems unlikely to change in the future.
The concise and action-oriented guide to Docker packaging for production
Docker packaging for production is complicated, with as many as 70+ best practices to get right. And you want small images, fast builds, and your Python application running securely.
Take the fast path to learning best practices, by using the Python on Docker Production Handbook.
Free ebook: "Introduction to Dockerizing for Production"
Learn a step-by-step iterative DevOps packaging process in this free mini-ebook. You'll learn what to prioritize, the decisions you need to make, and the ongoing organizational processes you need to start.
Plus, you'll join over 7000 people getting weekly emails covering practical tools and techniques, from Docker packaging to Python best practices.