I've written about zc.buildout and virtualenv before and how to use them both simultaneously, which I find to be useful for development on my own machine. I really admire both of these tools; I especially think that buildout is really great for projects where you want developers to be able to get your package running quickly without having to understand how python packaging works. (I use buildout for this purpose for one of my own personal projects, MediaGoblin, and I think it's served a wonderful purpose of getting new contributors up and going quickly.)

Anyway, in that previous blogpost about zc.buildout and virtualenv I erroneously suggested that virtualenv is best for multiple packages in development and zc.buildout is better for just one. I was rightly corrected that you can use the develop line of a buildout config file to specify multiple python packages. So this is what we've been doing for the last year roughly, running a meta-package with cc.engine checked out of git and the rest running out of python packages.

We've been doing packaging and releasing to our own egg basket for a while, and for the most part that has worked out, but our system administrator Nathan Kinkade pointed out that we don't really need packages, it's a bunch of extra steps to build, nobody outside of CC is using these packages, and it's a lot easier to rollback a git repository in case of an emergency than it is a python package.

That lead me to reconsider the way we're currently doing deployment and my growing feeling that maybe zc.buildout, while great for developing locally, really just isn't a good option for deployment. Whenever I want to pull down new versions of packages, I would run buildout. But buildout likes to do something which makes this period very, very painful: if for whatever reason it can't manage to install all packages, it tears down the entire environment. It removes ./bin/python, it removes all other scripts. I've found this to be highly stressful, especially because you never know if some package on PyPi is going to time out and then suddenly as punishment your environment no longer works, suddenly parts of creativecommons.org aren't running, and you start to have a minor panic attack as you rush to get things up again. That's not very great.

Anyway, I always stress out about this, which has lead me to adding coping mechanisms to our fabric deploy script:

This helps reduce my blood pressure somewhat, but anyway, we decided to move from buildout to virtualenv for deployment. Actually, there's not much more to say; it only took a couple of hours to make the switch and there really wasn't anything special to say about it. It just works and generally seems a lot simpler.

In short: buildout is pretty great. If you're looking for an option to make it really, really easy for people who want to try out your project to get something working or start contributing, it's the closest the python world has to an interface as simple as (or simpler than) ./configure && make. But as for deployment... especially if you'd like to do code checkouts of your main packages, just go with virtualenv.