Rawpixel.com - Fotolia

Mobile advertising network Billy Mobile bets the farm on Hadoop

Barcelona-based mobile advertising network Billy Mobile has opted to bet its business on the Hadoop stack with Hortonworks

Barcelona-based mobile advertising network Billy Mobile is betting its business on the Hadoop stack. The company, which began operations at the end of 2013, has signed up to use Hortonworks technology to support its growth.

Geoffrey Cleaves, business intelligence manager at Billy Mobile, said: “There are companies using Hadoop, but doing so departmentally. We are using it for 90% of our technology model and we use Kafka, Spark, Storm, Hive and Hbase. If we pulled out Hortonworks it would not be just a component of our business that would end, but the whole thing.”

The company will use Hortonworks Data Platform (HDP) for data collection, processing and analysis, replacing its previous architecture of PHP, MySQL and ElasticSearch.

“ElasticSearch is not meant to be a primary data store. No drivers exist inside it for business intelligence work, whereas Hortonworks does have the necessary ODBC drivers,” said Cleaves.

According to Cleaves, Billy Mobile is bridging a gap between publishers and advertisers exclusively on mobile platforms, “helping publishers monetise their assets and opening up the app ecosystem to carriers”.

It mostly operates in Hispanophone territories, but also has a presence in India and Asia, said Cleaves. It presently gets about 25 million visits per day, but it is aiming for billions per day as it scales up with the Hadoop technology.

The data team at the company use Tableau as their data visualisation tool. The team consists of two data scientists and Cleaves, with Cleaves focused on identifying the most commercially attractive patterns of usage. “The data scientists then turn that information into algorithms that make decisions as to what adverts to serve to which user at that time,” said Cleaves.

Their advertising customers include William Hill, Alibaba and Vibbo – a Spanish e-commerce site – and publishers include entertainment sites. Advertisers pay Billy Mobile each time a user downloads an app and the company pays the publisher a percentage of that.

Cleaves said: “We have built our business model around the core belief that mobile is revolutionising the internet at an unprecedented speed and scale. Hortonworks and Apache Hadoop have been selected to help us keep ahead of that internet revolution curve.

“From the start we were impressed by Hortonworks, from initial local support through to the fully open source, reliable and cost-effective HDP platform we have deployed.”

They began the Hortonworks programme of work in July 2015 and finished migrating their core technology stack early-December 2015 onto HDP 2.3.

To make its business work, Billy Mobile churns through tens of millions of records a day to determine the right offer for the right user using semantic analysis and machine learning.

The visitor’s profile and the displayed ad data is stored in Kafka while Storm streams the information into Hive and Hbase. A batch Spark process then analyses the Hive data and places a decision-making algorithm on the web front ends.

Cleaves said: “We know that when we make the jump to billions of daily impressions Hadoop can handle the load by expanding the cluster. We are further comforted with our decision because Hadoop runs (mostly) on Java and Java developers are in abundance. HiveQL is similar enough to SQL that our business analysts can easily handle the syntax. We can connect Tableau straight to Hive”.

They decided to work with Hortonworks, he said, because they liked its pure open source approach, which contrasts with alternatives, such as Cloudera and MapR. They are also getting professional services support from the supplier.

Read more about big data in mobile advertising

Read more on Business intelligence and analytics