Using devpi for local PyPI mirroring
Faster installs and testing from the cached lane
PyPI never goes down, right? Well, even if it doesn’t, running a fresh build on a Python library - in say, multiple test environments - or reprovisioning a virtual environment means dependence at least on a remote system.
A better solution is to use a local mirror. That’s where devpi comes in.
Cached mirroring, only what you use
Instead of attempting to mirror all of the packages on the package index, which seems to be the goal of a lot of package mirror systems, devpi primarily serves as a caching proxy to the Python package index. When you download a package using pip (or easy_install, as it were), devpi it caches a copy of the package so that the next time you request that package it comes from your local cache.
It does so by running a PyPI compatible server locally. You either specify this new index on the command line at package install time or - smarter - updating your pip configuration to use your new local index.
Testing and clean installs
One of the most helpful scenarios is testing Python packages. If I’m writing a Python library I’ll configure tox to test it against several different versions of Python for me. tox manages virtual environments for each test environment which means it’s download and installing dependencies for each environment.
This is multiplied when you start building a matrix against Python versions and dependency versions as well.
I run tests for the geocoding libraries I’ve written against Python 2.7, 3.3, 3.4, and as it fits, PyPy. For Django apps I matrix against various Django versions as well, so now we’re multiplying the number of test environments which means we’re multiplying the number of times every file must be downloaded.
Now if I run tox -r and rebuild the testing environments, all the
packages are downloaded directly from my own machine, rather than from
PyPI. Much faster.
Go get devpi. Now.
Check out the quickstart guide and try it today.