animind - Fotolia
More discernible this year, perhaps, is a dual convergence of big data and more traditional data management technologies, and of traditional, report-centric business intelligence with the more recent generations of analytics software, which have been more about self-service, data exploration, and so on – often drawing on the newer data stores, such as the Hadoop family and the NoSQL databases, but also on older data warehouses.
Financial services companies have been active here, and we see in this year’s top 10 re-insurer Munich Re combining a Hadoop data lake, SAS analytics and SAP Hana to create new cyber and other insurance services. In Dublin, Allied Irish Bank is blending artisanal data analytics work with a more “industrial” approach, drawing on Teradata data warehousing and a Hadoop cluster from IBM. And the Nordics’ Danske Bank has been using the Hortonworks distribution of Hadoop as a resource for its data science programme to combat fraud, among other use cases.
The open source parallel processing framework Spark, invented by the founders of Databricks, found fame as, in part, a replacement for MapReduce – an important element of the Hadoop data storage family in its earlier days. But this year the supplier launched a system that is neither a data lake nor a data warehouse, but, putatively, a synthesis of both – Delta. That, too, is an emblem of convergence of old and new in the data management arena.
Speaking to this broad theme of convergence have been some of the leading lights of data management and business intelligence and analytics, featured here: Michael Saylor, CEO, Microstrategy; Jim Goodnight, CEO, SAS; Doug Cutting, Hadoop co-creator and chief architect, Cloudera; and Douglas Farmer, formerly of Microsoft and Qlik and a much-respected thinker in the field of data and BI.
The concept of data for good is embodied in the anti-malaria work of the Malaria Control and Elimination Partnership in Africa (Macepa) at international non-profit health organisation Path. Here we see data analytics making a real difference to the welfare of humankind.
The Allied Irish Bank in Dublin has been muscling up its data analytics capability over the past three years, as part of a drive to gain a more intimate understanding of its 2.3 million customers.
Read how re-insurer Munich Re combines a Hadoop data lake, SAS analytics and SAP Hana to create new cyber and other insurance services.
Find out how Nordics bank Danske Bank has been using the Hortonworks distribution of Hadoop as a plank in its data science programme, which targets fraud, fills ATMs and aims to combat customer delinquency.
Read how the Path health organisation is fuelling its anti-malaria programme in Zambia and other African countries with data analytics from Alteryx, Tableau and others.
Donald Farmer is a well-respected figure in data analytics, with significant spells at Microsoft and Qlik. He is turning his mind to the ideologies that determine the character of companies.
London’s local authorities have a fraud problem with houses in multiple occupation (HMOs) and are using data science to combat it, with help from Nesta, the Greater London Authority and ASI Data Science.
At Spark Summit Europe 2017 in Dublin, the conference’s organising supplier and Spark inventor and distributor, Databricks, announced the Delta system, which will, it is said, combine data lakes and data warehouses.
Co-founder and CEO of MicroStrategy takes the long view on business intelligence and sees its universe expanding with mobile technologies.
SAS co-founder and CEO Jim Goodnight says the value of the vogue for artificial intelligence lies in massively parallel computing, as well as cheaper storage.
In a Q&A interview at Strata in London in June, the “father” of Hadoop, Doug Cutting, talked about the cyber security applications of the stack, as well as Hadoop’s evolution.