Debugging ImportError and ModuleNotFoundErrors in your Docker image
Your code runs fine on your computer, but when you try to package it with Docker you keep getting
ModuleNotFoundError: Python can’t find your code.
There are multiple reasons why this can happen, some of them Python-specific, some of them Docker-specific. So let’s go through a step-by-step process to figuring out what the problem is, and how to fix it.
Step 1. Are you using the right Python interpreter?
It’s easy to mistakenly end up with multiple Python interpreters in your Docker image. When that happens, you might install your code with interpreter A, but try to run it with interpreter B—which then can’t find the code.
Here’s a somewhat contrived example of how this happens:
FROM python:3.8-slim-buster RUN apt-get update && apt-get install -y python3 python3-pip RUN /usr/bin/pip3 install flask ENTRYPOINT ["python", "-c", "import flask"]
Note: Outside any specific best practice being demonstrated, the Dockerfiles in this article are not examples of best practices, since the added complexity would obscure the main point of the article.
Make sure your production software is packaged securely, efficiently, and quickly: Read the pragmatic, thorough, and concise Python on Docker Production Handbook.
You now have two Python interpreters:
/usr/local/bin/pythonis the Python interpreter provided by the Docker image. This is what the
/usr/bin/python3is the Python interpreter installed via
apt-get. This is what
Sometimes this happens less visibly, when you install a system package that depends on a
python3 system package, or try to install a library by doing
apt-get install python3-numpy.
This mostly happens if you’re (mis)using the official
python base image.
If you are:
- Don’t manually install Python.
- If you’re installing system packages, check if Python is one of the dependent system packages installed, and if so make sure you’re using the
- Install Python libraries using
Step 2. If relevant, are you successfully activating your virtualenv/conda env?
If you’re using an isolated environment like a virtualenv or Conda environment, you’re in a similar situation to the above: you have two different versions of Python, the system Python and the isolated environment. You want to make sure your code is installed and run from the same one.
Step 3. If it’s not
pip installed, make sure the code location is correct
This is where you need to start understanding how Python decides where you can import code from.
To begin with, Python has a series of standard directories where it checks for imports.
You can seem them by looking at
$ docker run -it python:3.8-slim-buster Python 3.8.3 (default, Jun 9 2020, 17:49:41) [GCC 8.3.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import sys >>> sys.path ['', '/usr/local/lib/python38.zip', '/usr/local/lib/python3.8', '/usr/local/lib/python3.8/lib-dynload', '/usr/local/lib/python3.8/site-packages']
We’ll ignore that first item, the
'', for now.
The next few are basically just to give you access to the Python standard library.
The last entry, ending with
site-packages, is where
pip will install packages.
So long as you’re installing your code with
pip, you’re all good.
It doesn’t matter what directory you’re in, you will be able to import it successfully.
$ docker run -it python:3.8-slim-buster bash root@3c19e2314f01:/# pip install --quiet flask root@3c19e2314f01:/# python -c "import flask; print('success')" success root@3c19e2314f01:/# cd /tmp root@3c19e2314f01:/tmp# python -c "import flask; print('success')" success
The problem usually arises when you’re not installing code with
pip, but rather just copying it into some arbitrary directory.
That’s where the first entry in
sys.path, the blank string
'', comes into effect.
'' means is “add the directory where the initial script you ran is to
Since that may be a little confusing, let’s look at two examples.
Let’s say we have two files,
main.py will do an
If they are in the same directory, everything will work:
$ tree . └── code ├── library.py └── main.py 1 directory, 2 files $ python code/main.py Successfully imported library.py $ cd code/ $ python main.py Successfully imported library.py
If they are in different directories, the import will fail:
$ tree . ├── code │ └── library.py └── main.py 1 directory, 2 files $ python main.py Traceback (most recent call last): File "main.py", line 1, in <module> import library ModuleNotFoundError: No module named 'library' $ cd code/ $ python ../main.py Traceback (most recent call last): File "../main.py", line 1, in <module> import library ModuleNotFoundError: No module named 'library'
Notice that your current working directory doesn’t matter. What matters is that the code you’re importing is in the same directory as the main script you’re running.
Either install all your code with
pip install, or make sure it’s all in the same directory.
python -m requires the code to be in the current working directory
If you’re using
python -m yourpackage or
python -m yourmodule, and you haven’t just
pip installed everything, then you need to follow the same requirements as step 3: the imported code needs to be in the same directory as the main script.
In addition, your current directory (which you can set in your
WORKDIR), must be the same directory where your code sits.
$ tree . └── code ├── library.py └── main.py 1 directory, 2 files $ python -m main /usr/bin/python: No module named main $ cd code/ $ python -m main Successfully imported library.py
If your code is a package, you need to be in the directory containing the package:
$ tree . ├── library.py └── myapp ├── __init__.py └── __main__.py 1 directory, 3 files $ python -m myapp Successfully imported library.py $ cd myapp/ $ python -m myapp /usr/bin/python: No module named myapp
Make sure your current working directory is the same as the code:
FROM python:3.8-slim-buster WORKDIR /code COPY myapp .
The short version
To recap, here are the suggested actions to take to prevent
- Make sure you only have one version of Python installed.
- If you’re using a virtualenv/conda env, make sure it’s correctly activated.
- Either install all your code with
- Make sure it’s all in the same directory.
- If you’re additionally using
python -m, make sure your working directory is the same as the directory the code resides in.
Want to learn more debugging techniques for common Docker packaging problems? There’s a whole chapter covering 13 different debugging techniques in Just Enough Docker Packaging, my introductory book for Python programmers.
Learn Docker packaging in just one afternoon
You need to start packaging your Python application with Docker, and you keep hitting errors, from connection refused to OCI runtime complaints, because you don't really understand how it all works.
Spend an afternoon learning both the fundamental concepts and the practical debugging techniques you need: read my concise, practical book on Docker packaging.
Free ebook: Introduction to Dockerizing for Production
Learn a step-by-step iterative DevOps packaging process in this free mini-ebook. You'll learn what to prioritize, the decisions you need to make, and the ongoing organizational processes you need to start.
Plus, you'll join my newsletter and get weekly articles covering practical tools and techniques, from Docker packaging to Python best practices.