Undocumented feature of the week: $PIP_DOWNLOAD_CACHE

Use Python? You should be using Pip. A replacement for easy_install, that supports uninstalling and plays nice with virtualenv. An apt-get for Python packages, if you will.

It has a marvellous undocumented feature. Set the environment variable PIP_DOWNLOAD_CACHE to prevent re-downloading the same packages repeatedly when setting up environments on the same machine:

> set | grep PIP
PIP_DOWNLOAD_CACHE=C:\Documents and Settings\jhartley\.pip_download_cache

> pip install mock
Downloading/unpacking mock
Creating supposed download cache at C:\Documents and Settings\jhartley\.pip_download_cache
 Downloading mock-0.7.0b2.zip (242Kb): 242Kb downloaded
 Storing download in cache at c:\documents and settings\jhartley\.pip_download_cache\http%3a%2f%2fpypi.python.org%2fpackages%2fsource%2fm%2fmock%2fmock-0.7.0b2.zip
Successfully installed mock

> pip uninstall mock
 Successfully uninstalled mock

> pip install mock
Downloading/unpacking mock
 Using download cache from C:\Documents and Settings\jhartley\.pip_download_cache\http%3A%2F%2Fpypi.python.org%2Fpackages%2Fsource%2Fm%2Fmock%2Fmock-0.7.0b2.zip
Successfully installed mock

(This text is copied from my unholy bastardised shell of choice at work, Windows CMD shell with Cygwin binaries foremost on the PATH.)

Using the download cache like this is substantially faster. Exactly what you need if you’re continually setting up environments under various version of Python for testing or what-have-you.

The directory is created if it doesn’t exist. Network access is still required when installing like this, presumably for the version checks.

Thanks to the irrepressible fuzzyman for bringing this to my attention.

10 thoughts on “Undocumented feature of the week: $PIP_DOWNLOAD_CACHE

  1. Pingback: Python:Can pip use existing packages rather than downloading again? – IT Sprite

  2. I do use pip on Windows as a replacement for ‘easy_install’, and at that it works just fine for me, but I’m not making heavy use of virtualenv.

  3. @Ben Pip’s hg trunk (http://bitbucket.org/ianb/pip) received a lot of Windows attention in May. The full test suite now passes on Windows, and there’s a continuous integration server making sure it stays that way (http://ci.cloudsilverlining.org/view/pip/). The “maybe it doesn’t work on Windows” line has also been removed from the tip docs.

    There isn’t yet a released version with these fixes, but there should be soon. (FWIW, I think most of the Windows fixes were to test-suite platform sensitivities; by and large I think the latest pip release does work fine on Windows already).

  4. So, is Pip working well enough on Windows? The “Maybe it doesn’t work on Windows.” line on the website has always deterred me from trying it.

  5. @PaulH Hey. Good question. I’ll double-check that Richard’s code (which forms the vast majority of the game) is appropriately licensed, and add a download of it somewhere. Gimme a couple of days to unpack both laundry and my mind from europython…

  6. I do enjoy having a read throgh Tartley.com from time to time – much fun!!

    Tell me Mr Hartley – where on this site can I download your asteroid game … screenshot looks smart … now I just want to have firtle with it … :)

  7. @Marius Indeed, although pip’s download cache only goes halfway to buildout’s egg cache, IIUC (not a buildout user myself). It saves on bandwidth and download time, but not disk space. The package is still installed on disk separately within each virtualenv’s site-packages. Generally not a problem these days, though I do recall David Glick complaining a bit about it when experimenting with installing Plone via pip/virtualenv :-)

  8. You beauty!

    I use a virtualenvwrapper postmkvirtualenv hook to install ipython in every new env. ipython takes an inordinately long time to download (enough time to make a cup of tea and watch it go cold).

    cheers Mr Hartley!

Leave a Reply