Twitter shares MySQL open source engineering 'at scale'

With what must surely be described as a relatively successful operational model under its wing, Twitter has decided to publicly release more of its work with the open source MySQL relational database management system.

While MySQL has at times come under fire in relation to its ability to scale and provide true enterprise performance connected to big data sets, Twitter has shown that it can operate with this technology at a “massively scaled” level.

Providing its code to the open source community could now arguably show other development projects a new way forward to using open data tools, languages and components in real world deployments.

NOTE: MySQL is named after its maker’s daughter My, with the SQL element standing for Structured Query Language. MySQL forms a part of the LAMP stack: Linux, Apache (web server), MySQL and PHP (sometimes also the Perl or Python languages).

This latest “open sourcing” by Twitter comes after the company has previously released other component technologies designed to plug into and sit on top of MySQL itself:

— the Gizzard middleware networking component,

— the FlockDB fault tolerant graph database (Twitter likes to create graphs of people’s relationships with other people) have both been released since 2010.


Reports suggest that Facebook and others have also showcased and shared MySQL work in the same vein as Twitter.

But, to date, one might reasonably argue that this Twitter MySQL fork (with its freedom from contracts or licences) could now open up even wider usage of the code donated in this instance.

Writing on the Twitter Engineers developer’s blog Jeremy Cole (@jeremycole) and Davi Arnaut (@darnaut) explain the following, “MySQL is the persistent storage technology behind most Twitter data: the interest graph, timelines, user data and the Tweets themselves. Due to our scale, we push MySQL a lot further than most companies. Of course, MySQL is open source software, so we have the ability to change it to suit our needs. Since we believe in sharing knowledge and that open source software facilitates innovation, we have decided to open source our MySQL work on GitHub under the BSD New license.”