Our software development future: "probabilistic" applications

IBM’s Lotusphere conference wrapped up today with an inspirational talk from Sir Tim-Berners Lee who spoke about the need to move to a more intelligent web.

Having already spent 20 year working on “engineering” the Internet, Berners-Lee now sees the need to progress to the “semantic web”, where computers understand more about the documents and data that we exchange over the wires.

Technologies included in this area include:

• contextual analysis of data – so we know more about what it really means

• natural language processing – and text analytics

• big data – and workload optimized processing

• … and also, processing carried out at the Teraflops level of computing

This leads us on logically (I hope) to mention IBM’s Manoj Saxena who is GM of the company’s Watson Solutions data analytics division. Watson was famously used on the US game show Jeopardy in a man vs. machine battle after IBM set out to build a computing system that rivals a human’s ability to answer questions posed in natural language with speed, accuracy and confidence

Before you ask, yes – Watson won when pitched against the show’s two most successful ever contestants.


Here’s our challenge. Saxena says that 90 per cent of the world’s data was created in the last two years and 80 per cent of that data is unstructured. This means that we need computers to achieve a new level of “reasoning”.

“I believe that ‘probabilistic’ applications will take over from ‘deterministic’ database driven applications in the future,” asserted Saxena.

What is a probabilistic application?

In basic terms, probabilistic logic would lead us to be able to create an application that might analyse natural human language based data and so, when asked a question about that data, be able to deliver an answer in the form of a hypothesis e.g. I (the computer) am 80% sure that answer A must be true due to the data analysis I have just performed.

While some may say that this progression towards probabilistic logic is nothing new — when interplayed and interconnected with the semantic web 3.0 Internet of the future I think it makes interesting food for thought.

Probabilisticly at least.