Andrea Izzotti - Fotolia

Top 10 information management stories of 2016

The use of big data technology has been a strong theme. Data visualisation and graph database technology came together in the Panama Papers, while Leicester City won the Premier League with data analytics

As in 2015, the maturation of big data technologies has been a strong theme this year, but with more stress on implementations, and the gaining of business value, beyond science projects.

The broad Hadoop family of technologies, including MapReduce replacement parallel processing framework Spark, has made inroads in the UK public sector, as well as companies such as Amex Global Business Travel, Barclays and Sky Betting. And Now TV is among those companies putting NoSQL database technology to use – in its case, to ease transmission of Game of Thrones.

Machine Learning is emerging as a central topic for the future. It is evident, here, in the stripe of cognitive computing represented by IBM Watson. Use cases are still incipient.

Data discovery suppliers, such as Tableau and Qlik, have captured much of the high ground in recent years, with the traditional players – such as SAP, IBM, Oracle, and Microstrategy, responding by coming back onto their visualisation turf. But there is also a new wave of “third generation” suppliers germinating in Silicon Valley.

The Panama Papers data disclosure of this year, which showed how the rich, global elite hide their wealth, was made possible through the use of data visualisation software, Linkurious, on the basis of the graph database Neo4j, as provided by Neo Technology.

Arguably the most interesting data analytics story of this year, however, was how it enabled Leicester City to win the English Premier League – a shock no one predicted.

Business Intelligence and data anlaytics are part of a broader data management story that we tell in Data management: fifty years of prospecting for business value, part of our Computer Weekly at 50 article series. It shows how business intelligence arose from data warehousing in the 1990s, and how it is developing today in the new, big data world of Hadoop and NoSQL databases. 

1. Apache Spark grows in popularity as Hadoop-based data lakes fill up

How Apache Spark is growing in popularity and finding real-time use across Europe, including in online betting and on railways, and with Hadoop.

2. NoSQL database technology finds use cases, but still minority sport

From managing water meter data, through Bitcoin and video data, to web publishing, NoSQL database technology is finding real use cases. But it’s likely to be stuck at 10% of the market for some time.

3. Hadoop starts to trumpet way through UK public sector

Read how big data technology Hadoop is starting to appear in patches of the UK public sector, including GCHQ, HMRC, the Home Office and the NHS.

4. San Francisco data analytics early-stage firms point to third-wave BI

California start-up and early-stage data analytics companies are positing technologies that might point beyond second-generation business intelligence, such as Qlik and Tableau.

 5. Foxy Leicester City FC won Premier League with data analytics

Leicester City Football Club was the surprise winner of this season’s English Premier League, thanks, in part, to data analytics.

 6. Panama Papers technology shows data journalism at its best

How the open source data analytics and visualisation software of the Panama Papers project empowers a new kind of journalism, according to the International Consortium of Investigative Journalists.

7. [email protected]: Data management – Five decades of prospecting for business value

Data management and business intelligence have been at the heart of business value creation for decades. Read about how Computer Weekly has tracked their promise and tribulations. The article concludes: “Whatever may be the technical shape of future data management architectures, the field can only develop as one of the main bearers of business value, coming from IT, for corporate organisations well into the future”.

8. For IBM, the answer is Watson. But what is the question?

IBM is promoting its Watson natural language processing analytics technology as it tries to move its core business beyond technologies that it pioneered, but which have become commoditised. Will it ever fly?

9. Amex Global Business Travel pools data lake to bolster ‘duty of care’

Amex Global Business Travel is investing in modern IT, including cloud analytics, to pivot its business towards serving the traveller for the entirety of their journey.

10. Cloudera users put pattern recognition at heart of new business ventures

Read how Cloudera users Barclays, BT, Sky Betting & Gaming and Markerstudy are finding new business development opportunities through the use of the Hadoop ecosystem.

Read more on Data warehousing