Read the Docs: A Completely Open Source Django Web Site

*
Accepted Session
Short Form
Scheduled: Tuesday, June 21, 2011 from 10:00 – 10:45am in B301

Excerpt

Read the Docs is a documentation hosting site for the community. It was built in 48 hours in the 2010 Django Dash. In January 2010 it had 100,000 page views, and increases daily. I will talk about all of the code to deploy and run a sizable Django site. We will go through the highlights and interesting parts of the code, as well as some of the lessons learned from the site being open source.

Description

Hosting open source documentation was a mess. The best-of-class solution for the Python world as uploading a tarball of html to packages.python.org or doing similar to upload to github pages. If you wanted to self-host it, that generally meant having a cron job that ran a shell script to pull your source code nightly. We set out to solve this problem using the current best of class tools that Django has to offer.

Read the Docs is the official documentation host for many open source Python projects. It is built around the Sphinx documentation toolkit. In the simplest form, we are a hosting provider for Sphinx documentation. However, we have added a lot of features to make this useful. These include:

  • Support for svn, hg, git, and bzr.
  • Post-commit hooks to automatically build documentation on commit
  • A custom Read the Docs styled Sphinx theme.
  • Full-text search across all projects.
  • Support for VCS tags and branches. (branches git only for now)
  • PDF generation for all documentation.
  • Editing of documentation that results in a pull request on github. (Bitbucket doesn’t have a pull request API)

Read the Docs has a lot of the standard parts of any website, and also some other intersting parts that are relatively unique. These include:

  • Subdomains
  • CNAME Support
  • Search using Solr and Haystack
  • Delayed task execution with Celery
  • Front end caching with Varnish
  • Deployment with Chef
  • Multi-server architecture
  • Monitoring with Nagios and Munin

This talk will consist of three parts. The first part is the origin story of the site, how and why it was created over a weekend by 3 people. Then I’ll talk about the technology involved as the site has grown. It started out as a very simple site, but as features have been added, it has gotten more complex. Finally I will discus some of the interesting outcomes that come from having a completely open source site, including security and community contributions.

Speaking experience

Speaker