WavebreakmediaMicro - Fotolia
The rise of technology powerhouses such as Google, Amazon, Facebook and others has resulted in plenty of obvious changes – the way we search, the way we shop and the way we communicate online, for example. But behind the scenes their huge operations have changed aspects of the technology industry as well.
One of those aspects is the database systems they use. For so long the domain of relational databases offered by the likes of Oracle, SAP, Microsoft and IBM, the past few years have seen the rise of NoSQL, NewSQL and graph databases, which are much better equipped to deal with the vast quantities of non-structured data that businesses are capturing, storing and managing.
There is nothing wrong with relational databases, of course; they do what they were designed to do very well. But as business needs change, a new style of database has begun to emerge, says Matt Aslett, research director, data platforms and analytics, at 451 Research.
“They were initially triggered by newer web properties, such as Google, Facebook, Amazon and LinkedIn, which have distributed data management challenges that relational database products were never designed to cope with. So these companies were forced to create their own databases, which met their need for distributed, highly available databases. That triggered a new breed of commercial products, based on the research carried out by Amazon and Google, in particular,” he says.
Using Neo4j to predict train delays
Neo Technology is the company behind the Neo4j graph database. Founded in Sweden in 2007 as an open-source project, the company’s technology focuses on connections and relationships between data. Rik van Bruggen, regional director at Neo Technology, says this can mean exploring previously unknown avenues.
“Relations between data elements become first-class citizens,” he says. “The connectivity is stored explicitly and not generated at query time. That means it’s much easier and faster and enables more possibilities to examine the connectivity between data. That enables lots of new business applications – you can start leveraging that connectivity to find unknown relationships between data.”
Because of this approach, the business drivers for customers of Neo4j tend to be very specific. Recommendation engines can use a graph database to return results much more quickly, for example, and delivery companies can use it to plan the most efficient route.
In Belgium, the national railway provider NMBS/SNCB is using Neo4j for a specific need; in this case as an impact analysis tool. When changes are made to train timetables there is the potential for the impact to be felt across the entire network, not just the local area.
Previously, NMBS/SNCB was relying heavily on local knowledge to calculate potential issues that could result in delays across the network. It now uses Neo4j to examine connections across the network to establish if a change to a local timetable would have any impact across the network as a whole. The change that would result in minimum disruption is then automatically implemented.
Using MarkLogic's NoSQL database to collect personal data
Another company that has implemented a new-generation database product as the result of a specific need is Austrian publishing company Herold Business Data. CIO Martin Kersch says the company, which publishes Austria’s version of the Yellow Pages, started using MarkLogic following an acquisition.
“In 2013, we had the challenge of integrating a recent acquisition. We’d bought the company’s Austrian arm – its staff and data, but not its IT system. So we needed to come up with a replacement rather quickly. Ordinary relational database models would have required a lot of data modelling on one side and an extensive application layer on the other side to provide what we needed, including constantly changing data sources,” he says.
The company selected MarkLogic’s NoSQL database after seeing a prototype being run by another company that was facing similar problems, which in Herold’s case was collecting personal data for direct mailings from many different sources.
Kersch says using a traditional relational database was certainly an option, but to achieve the same result would have involved a much higher financial investment from Herold.
That’s partly why newer database technologies are gaining such traction in the market at the moment, according to Matt Pfeil, chief customer officer of DataStax, the company that commercialises the Apache Cassandra NoSQL database.
“The core of any software is the database. Historically, storage hardware has been the blocker, but the per-gigabyte price these days is so low that companies can store unlimited data, for a better user experience,” says Pfeil. “But relational databases are architected to run on high-end, proprietary hardware. The next-gen database tech is here because it was built to scale-out on a cluster of low-end hardware, not high-end. That’s the primary motivation behind all this.”
Increasing data volumes drive need for next-generation databases
As detailed above, many of the primary business drivers for new-generation database technology revolve around solving specific problems. 451 Research’s Aslett believes NoSQL, NewSQL and the others are mainly used to complement traditional databases rather than replace them.
Adrian Carr, global vice-president of MarkLogic, says the change has been noticeable over the past couple of years. “The early adopter industries that were coming to us two years ago asking what NoSQL was are now coming to us with problems to solve,” he says.
But he adds that it is still doing a lot of educating in the mainstream industries, such as pharmaceuticals and life sciences. “Some of that is out of necessity, as companies are simply struggling with relational databases.”
The combination of lots of unstructured data and the near zero cost of storing it means the business drivers for NoSQL, NewSQL and other newer database technologies are a simple case of necessity. Traditional databases are far less equipped to deal with unstructured data, so if the volumes of data continue to grow, so too will the newer generation database industry.
Read more about NoSQL and NewSQL databases
- How logistics company DropShip Commerce went from SQL to NoSQL to NewSQL software to run its B2B platform, settling on a NuoDB database to help support business growth.
- Learn about NoSQL technologies and get advice from experienced users and consultants on how to select the right database software for your business needs and how to effectively deploy and manage NoSQL databases.
- E-commerce, social media and smart devices are all placing data demands on businesses that relational database systems cannot support at speed and reasonable cost. Although NoSQL technologies promise a solution, the term groups together a range of approaches to data management, each of which has its strengths and weaknesses depending on the problem corporate IT professionals are trying to solve.