The way business intelligence is deployed in companies must change. Avi Marco, head of corporate business intelligence at BSkyB, speaks to Cliff Saran about making BI more agile.
Marco reflects that BI professionals have been constrained by technology. "Sometimes the technology does not exist; sometimes it is too expensive. Most systems generally have poor performance and use disparate data management. Even if data can be joined, it still needs to be shifted across a network, which cannot handle it."
Even though data modelling is quite advanced - as Marco states, "We've got to the Holy Grail of slice and dice where you can almost second guess results" - BI is still not good enough.
He says that the rate of change is so fast that IT cannot keep up. "There is an adversarial relationship between the business and BI. It is madness." Getting BI projects done quicker does help, but according to Marco, "Speed alone is a hygiene factor. Business users will be happy for a while."
Marco says, "It is surprising how quickly users get used to a report taking 14 minutes to run compared with two and a half hours." IT needs to deliver quality BI quickly.
BSkyB has recognised the need to apply the agile software development methodology to the way it runs BI initiatives. Rather than take months to deliver a data warehouse for the business, Marco says it is now possible to start seeing benefits of a data warehouse within four weeks, and the project can be completed in 12.
So how does it work? "We bypass the analysis phase and ask the business what is the first big win," says Marco. For instance, the BI team delivered a margin analysis report using a Netezza data warehouse appliance for a Sky's new Voice service, within four weeks.
Agile means software development is fast, and therefore cheaper than traditional approaches, which involves more man-hours. As such, BSkyB can try new ideas without incurring huge IT costs. Marco says, "It costs far less to develop the code so I don't mind burning it." IT analysts work in Microsoft Excel. The Excel model is converted through a meta data layer built by BI specialist Sean Kelly Associates, into database queries.
In fact, by using Netezza, BSkyB has been able to focus on development, rather than database and storage management.
This is because Netezza, which was acquired by IBM this month, is a pre-integrated data warehouse appliance with built-in storage, optimised for high performance. It is designed to work out of the box.
"Since Netezza is simple to use, it doesn't require database administrators. The Netezza system is self-managing and I don't need to worry about storage management." As storage grows, BSkyB only needs to add an extra Netezza box.