Hortonworks: how to pin down data inside the 'Internet of Anything'

News this week sees Hortonworks finalise an agreement to acquire Onyara, Inc.

The rationale here is: Hortonworks as an open enterprise Hadoop company scooping up the creator of and key contributor to Apache NiFi.

NOTE: Apache NiFi supports scalable directed graphs of data routing, transformation and system mediation logic i.e. core ‘highway control’ for dataflow management inside IT systems.

Collect, conduct and curate

Hortonworks is hoping to make it easier to automate and secure data flows and to collect, conduct and curate real-time business insights and actions derived from data in motion.

As a result of the acquisition, Hortonworks is introducing Hortonworks DataFlow powered by Apache NiFi which is complementary to its own Open enterprise Hadoop platform, Hortonworks Data Platform (HDP).

What is the Internet of Anything?

This is the suggestion that this is a ‘new data paradigm’ that includes data from machines, sensors, geo-location devices, social feeds, clickstreams, server logs and more.

Many IoAT applications need two way connections and security from the edge to the datacentre — this results in a ‘jagged edge’ that increases the need for security but also data protection, governance and provenance.

These applications also need access to both data in-motion and data at-rest.

While the majority of today’s solutions are custom-built, loosely secured, difficult to manage and not integrated, Hortonworks DataFlow powered by Apache NiFi will simplify and accelerate the flow of data in motion into HDP for full fidelity analytics. Combined with HDP, these complementary offerings will give customers a holistic set of secure solutions to manage and find value in the increasing volume of streaming IoAT data.

What is NiFi?

Apache NiFi was made available through the NSA Technology Transfer Program in the fall of 2014.

In July 2015, NiFi became a Top-Level Project, signifying that its community and technology have been successfully governed under the Apache Software Foundation.

“Nearly a decade ago when IoAT began to emerge, we saw an opportunity to harness the massive new data types from people, places and things, and deliver it to businesses in a uniquely secure and simple way,” said Joe Witt, chief technology officer at Onyara. “We look forward to joining the Hortonworks team and continuing to work with the Apache community to advance NiFi.”

1 Hortonworks DataFlow FINAL.JPEG