Better business decisions were a key benefit of online
analytical processing, as reported in the third annual Olap Survey.
Industry expert Nigel Pendse analyses the results.
Online analytical processing (Olap) is a method of extracting
business information and allowing users to view it from different
"dimensions", such the sales for one product, in a chosen country,
over a defined sales period.
Over the past three years, the Olap survey has analysed the
benefits of this technology to business. In 2003 the survey
received answers from 2,897 respondents from 63 countries.
Of the respondents, about 40% had not yet considered Olap and about
10% were considering it but had not yet bought it. The remainder
were using more than 50 products, of which 10 were analysed.
As in recent years, the survey asked respondents about the extent
to which they had received eight potential business benefits using
Olap.
Two "soft" benefits - improved reporting and better business
decisions - were much more likely to have been achieved than "hard"
benefits such as reducing costs or headcount. It is likely that
more of the hard benefits would be achieved if firms set explicit
goals when undertaking business intelligence projects.
Weighted benefit scores can be used to assess the value of many
other project factors, including the products used, the amounts
spent, the data volumes, platforms, lead implementers, web
deployment rates and the criteria for product selection.
The rationale is that the best measure of project success is the
level of business benefits achieved rather than particular
technical achievements. If it is known which factors correlate
positively with benefit achievement, they can be favoured in
projects. Being feature-independent, this method allows the overall
success rates of different products to be compared.
Measuring project success
When investigating what affects business benefit achievements
levels, the most significant factor seemed to be the choice of
product. The users of some products reported markedly higher levels
of business benefits than others. The product with the highest
level of business achievement was Brio Intelligence (now Hyperion
Intelligence). The product with the lowest levels of reported
business success was SAP BW.
The same two products occupied the top and bottom slots in the
previous year, although the gap has widened in the latest survey
with the best products doing better than ever and the worst
slipping further behind.
Another area where there was a clear difference was in the time
taken for the initial roll-out of a project to users. Projects with
faster roll-outs seemed to have a much higher business success rate
than those taking longer.
Users spending £167,000 to £274,000 on licence fees appeared to
enjoy much more success, compared to those spending less than
£2,740. However the curve of business success versus licence fees
was not clear cut, and those spending more than £300,000 actually
reported a reduced level of business benefit. Even less clear was
any strong correlation between spend on external implementation
fees and business success.
Just about the least significant factor for Olap seemed to be the
choice of platform. Differences were slight, but Unix sites had
marginally lower levels of business success than others.
As in previous years, fast performance turns out to be a key
requirement, but one that is often not achieved. Oracle Express
users were the least likely to complain of poor query performance,
with only 7.7% of sites reporting this problem; the newer Oracle9i
Olap Option had too few active sites to measure. Microsoft Analysis
Services and Hyperion Essbase sites also had relatively few
performance complaints, with 10.5% and 11% respectively. At the
other extreme, 28.1% of Brio and 27.3% of SAP BW sites had a poor
query performance.
Many people think that query times should be below five seconds,
but only two products had median query times that beat this target:
Applix TM1 (4.2 seconds) and Microsoft Analysis Services (4.7
seconds). The worst was Brio, with a median of 33 seconds. The
overall median was 8.8 seconds, compared to 8.3 seconds in the
previous year.
Forecasting web services
The survey concentrated on asking respondents what their
organisations do, or what they have done in the past. It
deliberately asked very few questions about future plans, as the
survey is designed to gather information about what is actually
happening, rather than what people think might happen in the
future.
However, one area where the survey did ask users to forecast future
deployment plans concerned web and extranet projects. Of the 2001
sample, 67% expected to be at least 50% web-deployed by 2002, but
the actual rate in 2002 was just 37%.
The gap narrowed in 2002: 59% of the 2002 sample expected at least
50% web deployment in 12 months, but the actual rate was only 4%
higher, at 41%.
Again, the 2003 sample was slightly less bullish for 2004 than its
predecessors, but experience suggests that the true rate in 2004 is
likely to be in the mid forties, not the high fifties.
Exactly the same trend was found with extranet deployments, with
2002's sample predicting a dramatic increase from 12.6% to 31.1% by
mid 2003 where the actual increase was just 2.2%, instead of the
forecast 18.5%. The sites with the highest extranet deployments in
2003 used Microstrategy (with 25.7%), and the fewest (just 6.4%)
SAP BW sites deployed over an extranet.
The survey confirmed that web and extranet usage is much lower than
might be expected and growing a lot more slowly.
The Olap Survey 2003 showed many trends: the total sample showed a
much higher rate of Olap usage, project achievement rates were
rising and data volumes were increasing.
One striking observation was the amount of suppliers that were
independent when the survey was conducted that have since been
acquired. This included Brio, Cartesis, Comshare, Crystal Decisions
and MIS. Several others changed hands in 2002.
This may be part of an inevitable trend, but it is not necessarily
good news for users. In general, users of products where suppliers
have changed hands reported a lower level of success and worse
product support. Products from some of the larger suppliers scored
worse than those from smaller, independent firms.
Nigel Pendse is an independent industry
analyst and lead author of the Olap Report and the Olap
Surveys
More information
www.optimamedia.co.uk/?id=119
www.survey.com/olap
The Olap Surveys
The Olap Surveys are run independently, with no supplier
sponsorship or involvement. The survey recruits a sample of Olap
users, without access to suppliers' customer lists.
The Olap Survey 2003 collected data from 2,897 people in 63
countries. A total of 883 Computer Weekly readers responded.
www.olapreport.com