Python Packaging from Init to Deploy

Notes

A few years ago (circa 2009) I was working in the support department of a web hosting company. I had a number of tools and scripts that I used for day-to-day troubleshooting. Some of these were third-party packages that I installed via pip. Others were Python scripts that I had written myself or that my coworkers had shared.

We noticed that the ones I installed via pip were way easier to use. Every time I shared a script I had to tell the person using it to make the file executable and place it in their $PATH. I also had to remember to tell them to install any dependencies. I usually forgot to do at least one of those things so I had to help people get the scripts working every single time. Then once we got the thing working, someone would come up with an update and we'd have to manage sharing that update.

Clearly this wasn't working.

I needed to find out how to make my Python tools as easy to share and install as the third-party ones I installed. I thought it can't be that hard because look at all of the stuff that's been uploaded to PyPI? I did some searching to find how to make my Python scripts into installable packages and got a ton of answers. The problem was that a lot of them were contradictory. It was pretty clear that I needed a setup.py with some information in it but beyond that I didn't know what to believe. Should I use distutils? setuptools? distribute? I didn't know who to believe so I finally ended up copying a well-known project and just replacing their code and package details with mine. It worked for my purposes but was hardly the most obvious process.

Fast forward to 2014. I was at a new job and again had the need to make an in-house distributable Python package. This time when I searched I found an authoritative result! The Python community, realizing that special attention was needed for packaging, created the Python Packaging Authority and they created the Python Packaging User Guide. My PyOhio talk is about what I learned from the guide and additional tools and tips I found along the way.

Links

The following are links to more details about topics from the talk:

• Python Packaging Authority (PyPA)
• Python Packaging User Guide (PyPUG)
• Packaging Tool Recommendations
• Wheel vs. Egg
• install_requires vs. requirements.txt
• PEP440
• Versioneer

Addendum

I realized that I missed a couple points during the talk. Here are a few that I remember:

• The point of having at least minimal tests and docs from the very beginning is that other people are a whole lot more likely to submit a pull request adding to the docs/tests than they are to create them for you wholesale. Make it easy for people to help you!
• PyPI is also known as "The Cheese Shop."
• I like versioneer but there are definitely other options for single-sourcing the project version. A tool that takes a slightly different approach but seems useful is bumpversion.