Faster feature delivery, and happier developers and data scientists

There’s always more to learn in the world of Python, an ever-growing number of tools and techniques—but there’s only so much time in the workday. While your team is focusing on shipping the features that matter to your business, it’s easy for their technical skills to fall behind.

Out-of-date tools, stale techniques, and missing skills can kill your velocity. And if they feel their skills are becoming obsolete, your team members might move on to greener pastures elsewhere.

In contrast, investing in new skills and best practices can speed up your feature delivery—and keep your team happy and engaged.

“I thought it was really really good, and feedback from the team was that we would love to have you back again for another class.”

—Julian Berman, VP of Decisioning, Deloitte Digital

My training classes build on 20 years’ experience working with Python, and years of experience explaining and teaching complex software concepts:

• I’ve taught classes at Deloitte Digital in NYC, at Cisco offices in England and Norway, and to software engineers from VMware and Rackspace.
• Some of my recent talks at Python conferences:
  • Small Big Data: using NumPy and Pandas when your data doesn’t fit in memory (PyData NYC 2019)
  • Beyond cProfile: Performance optimization with sampling profilers and logging (PyGotham 2019)
  • Logging for Scientific Computing: Reproducability, Debugging, Optimization (PyCon 2019)
  • Big picture software testing: unit testing, Lean Startup, and everything in-between (PyCon 2017)

Interested in upgrading your teams’ skills? Get in touch.

Note: I prefer not to take money from teams working on projects for the military, prisons, fossil fuel extraction, surveillance, national security, or the like.

Available courses

Docker packaging for Python, part I: Basic packaging

Whether you want reproducible builds, the ability to build isolated environments, or you’re running software with frameworks like Kubernetes, you’re likely using Docker to package your software. Unfortunately, most online tutorials won’t help you understand how Docker works: when builds break, new users won’t know what to do or how to fix it.

This hands-on class will teach your team how Docker packaging works, not just the syntax but also the underlying model of how it works.

Syllabus

1. What is this Docker thing?
   1. Docker as client/server application.
   2. Containers as isolated processes: networking, filesystem, processes, and users.
   3. Running containers interactively.
   4. Port-forwarding.
   5. Images as self-contained executables.
2. Minimal packaging, from Python code to runnable image:
   1. Getting an image with Python.
   2. Installing dependencies.
   3. Installing the code.
   4. Setting the entrypoint.
3. Configuring your application to run in a container:
   1. Understanding port forwarding and its implications for interface binding.
   2. Persistent data: logs.
   3. Persistent data: volumes.
   4. Identifying network servers with environment variables and well-known hostnames.
   5. Command-line arguments.
4. Sharing images:
   1. Pulling vs. building.
   2. Understanding image names: image registry, image name, tag.
   3. Tagging images.
   4. Pushing images.
   5. The Docker Hub and official images.
5. Faster and smaller builds, the basics:
   1. The Docker build caching model.
   2. Improving caching by changing Dockerfile order.
   3. Reducing image size with .dockerignore.

Class format

The class utilizes live coding: I demonstrate live how to use Docker, and the students follow along on their laptops. Along the way I will make mistakes—some deliberate, some accidental—and show to work around them.

Each of the five sections then ends with an exercise to help solidify the learned concepts.

Interested in upgrading your teams’ skills? Get in touch.

Docker packaging for Python, part II: Production-ready packaging

Bad packaging can result in security and production problems, not to mention wasted time: waiting for slow builds, debugging the build system, and debugging issues caused by non-reproducible builds.

Your team wants to know best practices, so they can build and ship secure, fast images with confidence. To help you get there, I can teach your team how to build production-ready Docker images for Python applications, based on my extensive guide.

Syllabus

1. Deterministic, reproducible builds:
   1. Choosing a stable base image.
   2. Making images identifiable (tags, labels, files).
   3. Installing specific versions of Python dependencies.
   4. Maintaining two versions of Python dependencies, logical and pinned.
2. Securing your images:
   1. Don’t run as root.
   2. Dropping capabilities.
   3. Build secrets.
   4. Ensure security updates, and the problem with caching.
3. Startup and shutdown:
   1. Why bash is broken, and how to make it a little less so.
   2. Ensuring signal delivery for clean, fast shutdown.
   3. Health checks.
4. Preparing for failure:
   1. Writing smoke tests for your build.
   2. Debugging C crashes with faulthandler.
   3. A python prompt in your process with Manhole.
   4. Pre-installing useful packages, like glances and psutil.
5. Faster and smaller builds, using multi-stage builds:
   1. The problem with build tools, and multi-stage builds as a solution.
   2. Setting up virtualenvs in Dockerfiles.
   3. Multi-stage builds for Python.

Class format

The class utilizes live coding: I demonstrate live how to use these techniques, and the students follow along on their laptops. Along the way I will make mistakes—some deliberate, some accidental—and show to work around them.

Each of the five sections then ends with an exercise to help solidify the learned concepts.

Interested in upgrading your teams’ skills? Get in touch.

Software engineering for scientists and data scientists

Scientists and data scientists need to write software, but don’t always have the same software engineering background that software developers do. Based on my experience helping biologists with scientific computing, I can fill in these gaps in ways that are relevant to scientists’ particular uses of software.

Topics might include:

• Testing data pipelines.
• Performance for batch processing.
• Small Big Data—processing data that doesn’t fit in memory.
• And more!

Interested in upgrading your teams’ skills? Get in touch.

Performance optimization

Slow software is expensive—whether it’s high server costs, or batch processes that take too long. Teach your team the tools and techniques they need to figure out why their Python code is slow—from profilers to logging.

Interested in upgrading your teams’ skills? Get in touch.

Testing

Testing doesn’t just mean unit testing: everything from manual testing to code review has its place. I can help your team with the basics, or collaboratively figure out a strategy that meets your particular situation and goals.

Interested in upgrading your teams’ skills? Get in touch.