Building huge sites with MongoDB and Python

SourceForge has been using MongoDB since before 1.0. We’ve got a huge open source forge built with Python and MongoDB, and the experience of building a very large application with a schema free document database has been enlightening.

The power, speed and flexibility of MongoDB have been very rewarding. Schema free document based storage has made creating flexible ticket schema’s very easy. Documents can easily be copied making versioning tickets and other documets easy. But not having a schema has lead to bugs when an expected item is missing from the document.

This talk will focus on the real world schema design considerations we’ve faced when scaling MongoDB up to millions of writes per day (in one system), millions of page views per day (in another system) and to the needs of a full software forge (in a third system).

Beyond that I’ll talk about how we’ve worked around performance issues in our MongoDB related code, saturating the network with MongoDB traffic, and how a single group() query made our system administrators cry.