Business Intelligence software has stepped up a few gears, and now
provides analytic features that could only be dreamed of 10 years
ago. But can it live up to the hype, and really benefit business?
Over the years, there have been a number of approaches to provide
end users with the information they need to make decisions. They've
been variously labeled decision support systems, management
information systems and executive information systems.
Because of their cost, they tended to be restricted to higher
echelons, and often demanded both technical skills and knowledge of
sophisticated statistical and analytical techniques.
With the arrival of Web technology, companies have a mechanism for
deploying business intelligence (BI) throughout the organisation,
cheaply and with a low support requirement. "Intranet and extranet
adoption will bring enterprise decision support to a broader class
of users, both within and outside the enterprise," said Henry
Morris, the vice-president of IDC's Data Warehousing and
Information research programme.
Server-based BI applications can be accessed from any device with a
Web browser. But some major problems remain in integrating the
various sources of data users might need. Kevin Magee is sales and
marketing director of Information Builders, which has been
providing query and reporting tools for multiple data sources since
Larry Ellison was poor. "Most BI tools only access data stored in
relational databases," he says.
"This means more than half their data sources are inaccessible,
with 70% of all data still residing in legacy sources on
mainframes." Information Builders' WebFocus uses the company's
integration technology to access more than 80 data sources,
including legacy, relational, and ERP data.
New market
Steve Bavis, the business information
solutions manager at the ERP solutions company Intentia, said far
from being an up-and-coming technology, BI is already a "given"
with 95% of new customers. Bavis cites one of Intentia's customers,
Cummins Diesel, which, thanks to a BI system, discovered a new
market for its products among existing customers.
"As companies continue to consolidate they will find themselves
with dispersed information sources, often on multiple platforms,
structured, unstructured, object orientated, relational. It's a
real mess. However, suppliers are responding to the move toward
more collaborative relationships between companies by making their
software more open, in order to communicate with other systems and
make it possible to merge data."
'E-Researchers' Survey.com give Oracle top spot in the overall data
warehousing/BI software market, with over 50% of 600 study
participants naming it the leading vendor. Clustered behind Oracle
are Microsoft, IBM, and Cognos. "The market has been attracting
many new entrants, but the leading suppliers, including Oracle,
have been able to keep their positions strong," said Nancy Stewart,
Survey.com's programme director for enterprise computing research.
She said the flood of new offerings is expanding choice and
bringing the technology within reach of those who previously
couldn't afford it. Stewart said BI specialists Brio and Business
Objects are making good headway in the enterprise segment of the
warehousing/BI market.
"Microsoft holds sway in many small and medium-sized businesses,
and Oracle and IBM are strong in almost every segment, particularly
at the enterprise level, so second-tier suppliers have their work
cut out for them.'
Survey.com predicts that businesses will increase their annual
spending on data warehousing/BI solutions by 74%, and in Europe by
85%, in the next three years. "The warehousing and BI market will
weather the current economic downturn better than many other
segments of the technology market," Stewart asserted, although she
said spending on initial implementations is down by about 40% from
last year: "This reflects in part the current volatile market, but
also the change in the market. More, smaller sized solutions are
being implemented, with more packaged software and cost effective
components."
In a Gartner Group survey, more than 80% of IS professionals said
providing tools for users to access data directly is a strategic
priority. What they don't want to do is turn enthusiastic but
inexperienced users loose on the organisation's production systems,
with the risk of the fabled "query from hell".
One alternative is the data warehouse, a completely separate system
which extracts data from the production systems and holds it in a
form optimised for user queries. Data from many different sources
has to be extracted, validated and "cleaned", and transformed into
a suitable format for the data warehouse.
This can involve creating "multi-dimensional" databases which
contain the same information as the production databases, but which
can actually be much larger, because the information is presented
in different ways to facilitate analysis. The data has to be
regularly updated, or refreshed, at intervals that can vary from
weekly or monthly, to near-real time.
Many organisations have set up smaller "data marts", which address
the needs of specific business areas. These are much quicker to
build. However, by definition, they are incapable of giving the big
picture. There's also a risk that they will be set up in ways that
give different interpretations of the same data, or that they take
snapshots of corporate data at different times, so it's impossible
to get a consolidated view across departments.
According to one of the BI market leaders, Cognos, the fact that
current business cycles are six months or less makes data
warehouses unviable, particularly since they must include new
e-business data sources, as well as the legacy and ERP systems
which most data warehouse implementation and management systems
were designed for.
The best way to implement an enterprise business integration
solution, Cognos asserted, is to isolate a single critical area and
begin there, using data and business "dimensions" (such as time and
geography) that can be re-used in future datamarts.
This will ensure consistency, and make it possible to link the
datamarts to create a reliable big picture.
The success of data warehouses and marts depends on "metadata",
which consists of information about where data is held, which
reports contain it, when it was last refreshed, and when the next
extraction from the source databases is due.
According to Survey.com, IBM is the leading supplier of data
warehouse technology, with twice the market share of its nearest
competitor. Customers find the one-stop shop approach, including
consultancy and implementation from IBM Global Services, reassuring
in a market fraught with stories of disastrous multi-supplier
systems integration exercises.
IBM's solutions are based on the DB2 Universal Database, and
include the Data Replication Family, and DataJoiner for
multi-supplier database access. DB2 Warehouse Manager handles
business metadata, and the Data Warehouse Centre is used to manage
technical metadata.
Thanks in part to the introduction of smaller scale, lower cost
technology by Microsoft, the datawarehouse/datamart business is
doing well. "All segments of the data warehouse tool software
market are contributing to the overall market's healthy growth,"
said IDC senior analyst Dan Vesset.
"However, information access is leading the charge. The gradual
shift of revenue from warehouse generation and management to
information access tools is a factor of the maturation of data
warehousing. As buyers move from the initial stages of use to full
deployment, spending priorities will shift from capturing and
storing the data toward delivering the information to a wider range
of users."
One of the techniques used for getting information from data
warehouses, and sometimes operational databases, is data mining.
This differs from straightforward query and analysis, since it can
be used to find significant patterns that the organisation may have
been unaware of. Like BI in general, data mining is being brought
to the masses. First into the field with data mining built into the
database was Microsoft, with SQL Server 2000, which offers
algorithms for customer segmentation and predictive analysis,
followed by IBM, and more recently, Oracle, whose 9i database comes
complete with tools to analyse CRM data.
Predetermined criteria
IBM's DB2 Intelligent Miner
Scoring software enables database administrators to embed scoring
algorithms directly into DB2. Scoring is used to rank items
according to predetermined criteria, the most profitable or
credit-worthy customers, for example.
Intelligent Miner Scoring is also compatible with Oracle databases.
Partners such as Unica, Xchange, Angoss, the Centre for Data
Insight, and LogMetrix are using it to develop analytical
applications. The market for packaged analytical applications is
booming, according to IDC, and the fastest-growing analytic
applications sector is CRM.
Silvon, best known for data warehousing products for the iSeries,
offers the Stratum analytical application suite, which covers
customer analytics, marketing, sales, financial, procurement,
e-business and manufacturing. "The drive for pre-packaged
applications comes primarily from a need to increase the speed of
implementation, reduce costs, improve the return on investment and
put effective tools direct into the hands of users," said Silvon's
European managing director Mark Mahara.
Extra value
Mahara said such applications extract extra
value from existing ERP investments. "They can also reduce the need
for companies to change their ERP systems at all. Because they can
draw data from multiple ERP systems, they are ideal at providing a
bridge between companies within the same group. The fact that the
ERP market has not really recovered from Y2K is partly down to the
growth in analytical applications that are considerably cheaper and
quicker to implement, and offer considerably more value for
money."
Analytics also take pressure off IT, which no longer has to create
reports for users, Mahara said. "Users can control their analytical
environment themselves, without requiring any additional in-depth
training. In broad terms, timescales for implementing analytical
application are between one and three months."
The big ERP suppliers are moving into the analytic applications
market themselves. SAP, PeopleSoft and Oracle all have their own.
JD Edwards, working with BI tool supplier MicroStrategy, says its
Business Intelligence suite can be used with other suppliers' core
applications.
And with e-commerce booming, analytical applications are turning to
the Web. "Because of the need to understand buying behaviour and
build customer relationships, the basic server log based traffic
analysis applications that once dominated the market are being
replaced by more sophisticated web site analysis applications,"
said IDC's Vesset.
The focus is moving from delivering BI via intranets to Internet
"portals", essentially virtual spaces where enterprises meet their
customers, suppliers and employees. Modelled on public services
like Yahoo, they can provide "personalised" access to information,
based on an individual's role and status, or location. They support
collaborative work, enabling people involved in projects, perhaps
across companies, to share information.
"IT has historically been unable to provide an effective organising
principle for the desktop, that co-ordinates multiple data sources,
processes and people," said Ovum principal consultant David Wells.
Web based publishing and corporate intranets are adding to
information overload by capturing and distributing ever-larger
amounts of information from multiple internal and external sources.
"The overall result is a fragmented view of information, and
fractured business processes within the organisation," Wells said.
"Portals represent a radical transformation of corporate
information management, where the emphasis has shifted from
creating and storing information in databases and repositories, to
describing what information is available and relevant, and making
it easily accessible to end users in a single interface."
IBM's Enterprise Information Portal (EIP), based on WebSphere, is
described as "a foundation for access, not the interface". In other
words, it's middleware, which requires user organisations to build
or buy a custom portal on top. BusinessObjects has an InfoView for
IBM EIP integration kit, to link IBM's middleware to its own portal
technology.
"Business Intelligence and Knowledge Management can't be
separated," said Barrie Pike, head of business intelligence at
business technology solutions provider Conchango, who sits on the
Microsoft Data Warehouse/Business Intelligence Partner Advisory
Council.
"Portals are set to replace intranets, as they let you combine BI
and KM, dealing not only in structured data, but also unstructured
information; the stuff in people's heads."
Tools market growth
"The overall information access tools
market will earn solid growth of approximately 21% compounded
annually through 2004," according to IDC senior analyst Dan Vesset.
"However, comparing the growth rates of each of the market's
sub-segments will reveal significant discrepancies. End-user query
and reporting, data mining, and online analytical processing will
be fast growers; while executive information systems, spreadsheets,
and statistical and technical data analysis will all be
slow-growing markets."
Data warehousing project
The telco Energis is carrying
out a huge data warehousing implementation which, only 15% of the
way into the implementation, had already reduced customer churn by
1%. Energis' financial reporting time has been reduced from six
weeks to one day. DataMirror's Constellar Hub is used to integrate
data into the warehouse, which sits on a DB2 database and is driven
by an IBM SP2 massively parallel server.
Enterprise applications
In a survey of 300 companies
conducted by Delphi Group, top applications for Enterprise
Information Portals were: Knowledge bases and learning tools 13.2%;
Business process support 12%; Customer facing sales, marketing and
services 11.7%; Collaboration and project support 10.2%; Access to
data from disparate corporate systems 9.1%; Internal company
information 9.1%; Policies and procedures 7.1%; Best practices and
lessons learned 6.6%; Human resources and benefits 6.4%;
Directories and bulletin boards 5.8%; Identification of experts
4.6%; News and Internet 4.1%.
BA's e-BI strategy
British Airways uses Business
Objects e-business intelligence (e-BI) tools to deliver strategic
information worldwide, in support of Customer Relationship
Management, yield management, revenue management, flight
punctuality and baggage handling.
"We analyse routes, market share data and bookings through travel
agents and computerised reservation systems to calibrate routes and
schedules," says Peter Blundell, British Airways' knowledge
strategy manager. Understanding market dynamics enables marketing
campaigns to be launched into areas likely to be fruitful. "You
want to segment your market so you don't flood the market with
deals that nobody's going to take up," Blundell says.
"Getting the right offer to the right group is very important.
Customer intelligence enables us to understand what our booking
profiles and customer profiles look like, so that we can make the
right offer to the right person. Blundell estimates that BA's BI
has brought $100m in cost savings and new revenues.