Deal Group Media connects online advertisers with affiliates - highly trafficked websites - with whom they place banner advertisements and pay for click-throughs or closed sales. As a pioneer of affiliate marketing, DGM knows the model depends on business intelligence and tracking to ensure advertisers get the optimum return on their investment.
"People traditionally think of business intelligence as an internal tool that looks inwards at products and processes. Our take on business intelligence is slightly different. We want to offer it to our clients. They want to know how to make the most efficient use of their advertising revenue online," says Carl Davis, chief technical officer at DGM.
The company markets through its own affiliate network, search engine marketing and major portals such as MSN or Yahoo. Clients want a mix of all three online marketing routes, and want to know where best to spend their advertising budget. Specifically, they need to know how their customers navigate online channels and arrive at the point of purchase.
"For example, a consumer purchasing a specialist product might go to Google, or visit a price comparison website. We need to show whether they have arrived via an affiliate link or a banner in Lycos. We are an industry that is dominated by 'last click' thinking but it is not a good indicator of the true user journey and the thought processes that lead up to a purchase. We need to better educate people through the use of business intelligence," says Davis.
The IT team decided to write its own business intelligence application, Integra, which receives information from three different ad-serving systems. Two of these platforms are external - Helios Adtech serves volume banners to portals and DCStorm delivers listings in the search engines. Additionally, Integra pulls in data from DGM's own affiliate ad-serving and tracking system, which uses MySQL.
Developers wrote Application Programming Interfaces in Java to call data from the three ad-serving platforms, which is requested every hour and loaded into a MySQL database. Data is generated by tags, located on portals and affiliate sites that identify the advertiser, the website and even the specific banner can be interrogated with questions such as "how many banner clicks have there been today?".
Tying these facts together and providing clients with business intelligence that exceeds the "last click mindset" is the sales order ID. This unique reference number is tagged - invisibly - on every page that a consumer visits leading up to a purchase. "We pull data from all three systems, match under the order ID - usually we will find the ID in up to three channels - and see the route they took to a sale," says Davis.
The advantage of interrogating systems over the internet is that web service definition language is universal and makes for easy data collection. "Once they are published, they contain meta data as to how they should be retrieved," says Davis.
The downside is that each of the systems publishes data in a different format. So, responding to "how many banner clicks today?" some systems list client details while others provide a summary by date and time. The Java code is written to deal with each Application Programming Interface and convert the data into the format required by DGM's business intelligence application Integra.
Three key pieces of information are collected for analysis: numbers of clicks a banner has received, the number of page impressions for client collateral, and the number of sales recorded for products. From this, the system can further manipulate the data and work out metrics more valuable to the client, such as click-through rates and on-site conversion.
Davis decided to try out two open source tools, Pentaho and Jasper - both native to Linux - to sit on top of the MySQL and do the querying. A trial was a good opportunity to test drive reporting, analysis, dashboard, data mining and workflow capabilities and how well it could adapt to DGM's unique requirements.
Although his developers like the flexibility of Pentaho, Davis says he has concerns with Pentaho and Jasper, primarily regarding the shortage of skills in the UK. "If you type Pentaho or Jasper into Jobserve, you do not get many listings. It is a really good benchmark of how much demand (and availability of skills) there is out there" he says.
Before Davis makes a final decision on business intelligence tools, he needs to be reassured that they have a good support environment. "Online feedback is OK but you also need to talk to a human to solve a problem, now and then."
Additionally, DGM has developed a graphical user interface to let customers examine data and generate reports. This was written with PHP, the open source web scripting language, and has had good feedback in its initial run. The future plan is to use a business intelligence layer instead of coding each report by hand, which should be more flexible and powerful.