MongoDB: the modern application's polymorphic data journey

Question: What is a technology conference without an appearance from Ray Wang of Constellation Research? R Ray Wang (王瑞光) @rwang0

Answer: Not much, he seems to be at every one of them of note.

But before Mr Wang and the ‘fireside chat’ experience at MongoDB World 2015, let’s go to the keynote.


CTO and co-founder of MongoDB Eliot Horowitz reminded us that with MongoDB 3.0 we saw “document level concurrency” introduced — this technology has now been enriched through the acquisition of the Wired Tiger storage engine.

NOTE: Wired Tiger will now be default in MongoDB 3.2 going forward.

Hang on, did you say document level concurrency?

Yes indeed, MongoDB allows multiple clients to read and write the same data and in order to ensure consistency, it uses locking and other concurrency control measures to prevent multiple clients from modifying the same piece of data simultaneously.

“Together, these mechanisms guarantee that all writes to a single document occur either in full or not at all and that clients never see an inconsistent view of the data,” said the firm.

The news on day two of this show hinges around the following statement:

For the First Time, Modern Application Data from MongoDB Can be Easily Analysed with Industry-standard Business Intelligence and Visual Analytics

What does that mean?

The company announced a new connector for BI and visualization, that connects MongoDB to industry-standard business intelligence (BI) and data visualization tools.

Designed to work with every SQL-compliant data analysis tool on the market, including Tableau, SAP Business Objects, Qlik and IBM Cognos Business Intelligence, the connector is currently in preview release and expected to become generally available in the fourth quarter of 2015.

Users will be able to analyse the new data being managed in MongoDB for what the company calls their ‘modern applications’, along with the traditional data in their SQL databases and spreadsheets using BI and visualisation tools deployed on millions of enterprise devices.

NOTE: Previously, organisations had to move data from MongoDB into relational databases for analysis and visualization, resulting in added time to insight, cost and complexity.

With the emergence of new data sources such as social media, mobile applications and sensor-equipped “Internet of Things” networks, organisations can extend BI to deliver real-time insight and discovery into such areas as operational performance, customer satisfaction and competitor behaviour.

“For many years now, MongoDB has provided extensive aggregation and analytical capabilities native to the database. However, users have been unable to capitalise on the rich ecosystem of SQL-based data analysis and visualization tools. With this new connector, MongoDB opens up a huge new realm of possibilities for everyone from executives to business analysts to data scientists to line of business staff,” said Horowitz, Co-Founder and CTO of MongoDB.

“This is a lightweight connector with heavyweight capabilities. The possibilities are endless,” he added.

Initially, MongoDB has been working with joint customers of Tableau Software to define the critical feature set and to validate the performance and scalability of the integration.

Schema good, polymorphic schema better

Horowitz reminded us that we often think about MongoDB as a schema-less database, but this isn’t really true i.e. it does have schemas, all databases have to have schemas, it’s just that MongoDB has dynamic schemas that have the ability to change and morph – and this is why we call them polymorphic schemas.

Remember… use of NoSQL dynamic polymorphic schemas means that dissimilar data sets can be stored together and this is good for the new world of unstructured big data.

It’s like there’s this whole modern ‘polymorphic application data’ journey that we are traveling on… we need to deconstruct this statement and analyse it further.

That’s number-Ray Wang

Back to where we started and our fireside chat hosted by Ray Wang who had to juggle with the presence of the highly entertaining Philip Mudd.

Mudd is a published author and former deputy director of the CIA’s counterterrorist center and FBI National Security Branch. His comments on the way his team was using the kinds of data analysis we are talking about here in relation to MongoDB left the audience well entertained and somewhat stunned.

He covered a good deal of information on the NSA… and how Al Queda had worked with its funding operation.

The real secret to big data?

“We don’t need to summarise all the crap we are collecting, we need to summarise what we can do with this crap so that we start with the decisions and work our way backwards,” said Mudd.

“If you are a great analyst you realise that 90% of what you know if of no use because it doesn’t lead to a point of ‘decision advantage’ now,” added Mudd.

Insight is context plus decision plus predictability said Wang… there’s a whole additional story to tell here.