Herding 1000 Cats^H^H^H^HDevelopers: What Openstack has done to scale contributions horizontally

Short Form


Since its small beginnings in 2010 at the Austin design summit, Openstack's developer base has grown tremendously and is now more than 1000 strong. A variety of tools have been put in place to accommodate this growth including design summits, code review, continuous integration on steroids, and more.


How do you scale a project to thousands of developers? Openstack isn’t unique in tackling this problem, but has approached it in its own way. In short we put contributors in rooms every six months so that they can plan the next release, do pre merge code review on every commit, and run as many automated tests as possible to ensure quality. All of this is done using only open source tools.

The twice yearly design summits make sure individuals with different agendas can agree on a plan for the next six months. Additionally they help build relationships, important when typical interaction is via email or IRC. Every change is code reviewed making the contribution process egalitarian with no secret back doors. This is particularly important for welcoming new contributors. We test everything, more than once. On a busy day more than twenty thousand test jobs are run providing crucial feedback to contributors and code reviewers.

This talk will cover how these tools fit together and how they can be implemented in projects of all sizes. We will also spend some time looking ahead to the future of scaling cat herding.


Code Review, Continuous Integration, collaboration

Speaking experience

I have not given this talk before. I recently gave a talk at Linux Conf AU 2014 on how Openstack processes continuous integration logs to build better software. Video can be found at http://mirror.linux.org.au/pub/linux.conf.au/2014/Friday/116-Processing_Continuous_Integration_Log_Events_for_Great_Good_-_Clark_Boylan.mp4 Slides can be found at http://docs.openstack.org/infra/publications/processing-ci-log-events/#(1)

I have also run design sessions at Openstack developer summits.


  • Clark Boylan



    A former Portland resident Clark moved to Seattle in search of cloudier weather. Having found the clouds, Clark is now a core member of Openstack’s infrastructure team where he helps build and run Openstack’s developer tools. When not diagnosing a cluster of 800 test slaves, Clark can often be found in Hillsboro brewing beer in his brother’s back yard.