Notes from the keynote: Apache Big Data & ApacheCon Core Europe 2015

Conferences come and go, but Apache: Big Data Europe and its sister event ApacheCon Core Europe 2015 is kind of special… as it’s a pure thoroughbred user conference.

1 Apach.JPG

Compatibility & quality paramount

As already reported on the Computer Weekly Open Source Insider blog, this event is dedicated to the vendor-neutral work of the Apache Foundation and its focus on products that power petabytes of data, teraflops of operations and billions of objects.

Keynote: open data advancement

The keynote panel at the event itself was called the ‘ODPi — Advancing Open Data for the Enterprise Panel’ and it featured – Anjul Bhambri, IBM; Konstantin Boudnik, WANdisco; Owen O’Malley, Hortonworks; Roman Shaposhnik, Pivotal — it was moderated by C. Craig Ross, director of developer programs, The Linux Foundation.

A reference implementation specification has been developed to produce a more formalised documented approach to using (and developing with) the technologies involved here — as usual, you can expect to be able to find these technologies available on GitHub.

Although the panel acknowledged that although there has always been a lot of opportunity with Apache Hadoop, there has also been a lot of fragmentation — so this event is a good opportunity for users to be able to find out which elements of the technology work together well.

Let’s get to the core

Customers need to figure out which distribution of Hadoop they need to go and we need to make it easier to make that decision easier — this way, those people involved with the project can focus on putting efforts into the Hadoop core and making the technology itself better.

At the point you might be wondering what ODPi is…?

This is a Linux Foundation Collaborative Project, which are independently funded software projects that harness the power of collaborative development.

“ODPi is a useful downstream project for the community to work on a common reference platform and set of technologies around Hadoop,” said Jim Zemlin, executive director at The Linux Foundation. “We’ve seen this model work with open source technologies experiencing rapid growth and know it can increase adoption and open up opportunities for innovation on top of an already strong Hadoop community.”

The panel confirmed that ODPi is in no way supposed to be a replacement for Apache and that the project itself is fully connected to the wider upstream projects here.

NOTE: ODPi uses an open governance model that is led by a community of developers who will form a Technical Steering Committee (TSC) based on expertise and value of contribution.

When asked what the projects goals are in the coming months, the panel confirmed that they are looking to finesse the finer details in the current release of the software.

This is clearly an important summit for the Apache team and a good opportunity for them to get together and agree on priorities (and key facilitating and steering technologies) as we now go forward — thinking about COMPATIBILITY will be a key part of this process i.e. the team will be looking to make the maximum amount of other software work on Apache Hadoop without unnecessary alterations.

In the future… speakers and audience members alike said they would like to see RFP (request for proposals) start listing a question point to assess whether particular technologies are ODPi compliant — it’s a good goal to aim for.

In terms of barriers to entry for Hadoop in the enterprise, the panel recognised that some obstacles will exist in some firms — but what matters is that we make technical resources available (with technical compliancy) to the people that want to use these technologies today without trying to stipulate draconian usage rules upon these development teams.

Governance will always be strong

Other elements of the discussion here focused on the importance of open governance — there’s a huge difference between a project that has been developed under the auspices of Apache as opposed to one that has just been “dumped” on GitHub.

Whether that’s a cruel term (dump) or not, the point is well made… The Apache Foundation continues to uphold its principles and for that we must applaud it.

2 Apach.jpg