The state of business intelligence in 2003

Better business decisions were a key benefit of online analytical processing, as reported in the third annual Olap Survey....

Better business decisions were a key benefit of online analytical processing, as reported in the third annual Olap Survey. Industry expert Nigel Pendse analyses the results.

Online analytical processing (Olap) is a method of extracting business information and allowing users to view it from different "dimensions", such the sales for one product, in a chosen country, over a defined sales period.

Over the past three years, the Olap survey has analysed the benefits of this technology to business. In 2003 the survey received answers from 2,897 respondents from 63 countries.

Of the respondents, about 40% had not yet considered Olap and about 10% were considering it but had not yet bought it. The remainder were using more than 50 products, of which 10 were analysed.

As in recent years, the survey asked respondents about the extent to which they had received eight potential business benefits using Olap.

Two "soft" benefits - improved reporting and better business decisions - were much more likely to have been achieved than "hard" benefits such as reducing costs or headcount. It is likely that more of the hard benefits would be achieved if firms set explicit goals when undertaking business intelligence projects.

Weighted benefit scores can be used to assess the value of many other project factors, including the products used, the amounts spent, the data volumes, platforms, lead implementers, web deployment rates and the criteria for product selection.

The rationale is that the best measure of project success is the level of business benefits achieved rather than particular technical achievements. If it is known which factors correlate positively with benefit achievement, they can be favoured in projects. Being feature-independent, this method allows the overall success rates of different products to be compared.

Measuring project success

When investigating what affects business benefit achievements levels, the most significant factor seemed to be the choice of product. The users of some products reported markedly higher levels of business benefits than others. The product with the highest level of business achievement was Brio Intelligence (now Hyperion Intelligence). The product with the lowest levels of reported business success was SAP BW.

The same two products occupied the top and bottom slots in the previous year, although the gap has widened in the latest survey with the best products doing better than ever and the worst slipping further behind.

Another area where there was a clear difference was in the time taken for the initial roll-out of a project to users. Projects with faster roll-outs seemed to have a much higher business success rate than those taking longer.

Users spending £167,000 to £274,000 on licence fees appeared to enjoy much more success, compared to those spending less than £2,740. However the curve of business success versus licence fees was not clear cut, and those spending more than £300,000 actually reported a reduced level of business benefit. Even less clear was any strong correlation between spend on external implementation fees and business success.

Just about the least significant factor for Olap seemed to be the choice of platform. Differences were slight, but Unix sites had marginally lower levels of business success than others.

As in previous years, fast performance turns out to be a key requirement, but one that is often not achieved. Oracle Express users were the least likely to complain of poor query performance, with only 7.7% of sites reporting this problem; the newer Oracle9i Olap Option had too few active sites to measure. Microsoft Analysis Services and Hyperion Essbase sites also had relatively few performance complaints, with 10.5% and 11% respectively. At the other extreme, 28.1% of Brio and 27.3% of SAP BW sites had a poor query performance.

Many people think that query times should be below five seconds, but only two products had median query times that beat this target: Applix TM1 (4.2 seconds) and Microsoft Analysis Services (4.7 seconds). The worst was Brio, with a median of 33 seconds. The overall median was 8.8 seconds, compared to 8.3 seconds in the previous year.

Forecasting web services

The survey concentrated on asking respondents what their organisations do, or what they have done in the past. It deliberately asked very few questions about future plans, as the survey is designed to gather information about what is actually happening, rather than what people think might happen in the future.

However, one area where the survey did ask users to forecast future deployment plans concerned web and extranet projects. Of the 2001 sample, 67% expected to be at least 50% web-deployed by 2002, but the actual rate in 2002 was just 37%.

The gap narrowed in 2002: 59% of the 2002 sample expected at least 50% web deployment in 12 months, but the actual rate was only 4% higher, at 41%.

Again, the 2003 sample was slightly less bullish for 2004 than its predecessors, but experience suggests that the true rate in 2004 is likely to be in the mid forties, not the high fifties.

Exactly the same trend was found with extranet deployments, with 2002's sample predicting a dramatic increase from 12.6% to 31.1% by mid 2003 where the actual increase was just 2.2%, instead of the forecast 18.5%. The sites with the highest extranet deployments in 2003 used Microstrategy (with 25.7%), and the fewest (just 6.4%) SAP BW sites deployed over an extranet.

The survey confirmed that web and extranet usage is much lower than might be expected and growing a lot more slowly.

The Olap Survey 2003 showed many trends: the total sample showed a much higher rate of Olap usage, project achievement rates were rising and data volumes were increasing.

One striking observation was the amount of suppliers that were independent when the survey was conducted that have since been acquired. This included Brio, Cartesis, Comshare, Crystal Decisions and MIS. Several others changed hands in 2002.

This may be part of an inevitable trend, but it is not necessarily good news for users. In general, users of products where suppliers have changed hands reported a lower level of success and worse product support. Products from some of the larger suppliers scored worse than those from smaller, independent firms.

Nigel Pendse is an independent industry analyst and lead author of the Olap Report and the Olap Surveys

More information

The Olap Surveys   

The Olap Surveys are run independently, with no supplier sponsorship or involvement. The survey recruits a  sample of Olap users, without access to suppliers' customer lists.  

The Olap Survey 2003 collected data from 2,897 people in 63 countries. A total of 883 Computer Weekly readers responded.

Read more on Managing IT and business issues