Twitter engineers have developed a new back-end search for Twitter.com with open source index search engine Lucene, to improve the micro-blogging site's search capability.
The new technology replaces Summize's relational database search engine, which was acquired by Twitter, with an inverted index search engine library written in Java. Software engineers re-wrote parts of the core in-memory data structures to meet real-time search demands.
"Scaling the old MySQL-based system had become increasingly challenging," said Twitter developer, Michael Busch, in a blog post.
"Twitter is real-time, so our search engine must be too. In addition to these scalability requirements, we also need to support extremely low indexing latencies (the time it takes between when a Tweet is tweeted and when it becomes searchable) of less than 10 seconds," said Busch.
"Our new indexer could also index roughly 50 times more Tweets per-second than we currently get," he added.
Twitter recently revamped its website to be faster, easier to use and more capable of handling photos and videos in the biggest overhaul in its four-year history.