Choosing middleware to knit together disparate systems can be one
of the biggest headaches in IT. But getting it right can also be
one of the biggest opportunities to reap competitive advantage.
Philip Hunter reports
If you think your operating system constitutes the most critical
software in your enterprise IT, think again. Middleware looks set
to become the IT battleground of the future, the facet of your
infrastructure from which you will need to thrash out business
advantage. While the operating system is fast becoming a commodity
item, the middleware tying applications and processes together is
still an ill-defined range of software components, requiring
considerable customisation and having no commonly agreed
definition. Unfortunately, it is also far harder to get right.
The major analyst firms are predicting a rosy future for
middleware. Gartner forecasts that middleware software will account
for 28% of IT expenditure by 2004, compared with just 1% in 1999.
This latter figure of 1% might seem puzzling given that integration
has been a significant overhead in large IT projects for some time,
but it only includes the software, not the internal IT effort of
making it work. Indeed according to Ed Wrazen, marketing director
of middleware consultancy CommerceQuest, integration already
accounts for 30% to 40% of the cost of a typical IT project, and
often as much as 70%.
Middleware software itself is supposed to ease the integration
burden, by providing at least some of the functionality
off-the-shelf to link applications together and provide more
consistent access to information. In practice though middleware can
create as many integration problems as it solves, if care is not
taken to adopt a coherent strategy for enterprise application
integration (EAI). For just as most large enterprises have a
variety of different ERP systems, perhaps because of acquisitions
or devolution of IT to local departments, so they have middleware
from more than one supplier.
And yet, as Hanafi Houbart, senior consultant in e-commerce at
Unipower Systems, points out, changing middleware on any scale is
usually difficult and expensive. "In practice no one ever changes
established middleware unless either the underlying operating
system [bottomware], or overlying applications [topware] are also
changed en masse, or the middleware itself has failed to establish
well," says Houbart. "That's why middleware products tend to have
longer product histories than front-line applications - they just
keep getting upgraded."
If there is a time to change your principal middleware supplier
though, for whatever reason, it is now. Within a few years
middleware will be even more inextricably linked with key platforms
and applications, making change yet harder. In theory it should be
possible to change horses if the middleware adheres to common
standards, but in truth the relevant standards do not yet exist and
in any case are hard to crystallise in a continuously changing
field. A major problem, according to Kay Hammer, chief executive
officer of integration software firm ETI, is that applications are
tied in with the middleware via application programming interfaces
(APIs) in a way that is hard to standardise and needs completely
reworking when you change supplier. This can be a complex process
with the danger of introducing errors or unanticipated changes to
processes.
"So while over time there will be some standard XML message
format that would facilitate moving from one middleware supplier to
another, to some extent while you're putting calls into your
programs that directly reference their APIs, then you're really
putting yourself at some risk," says Hammer.
To avoid becoming effectively locked in to their primary
middleware, enterprises must build their own software shell between
their applications and the back end or "bottomware" systems such as
the database. The most enlightened organisations have already done
this, says Hammer, who cites the case of one customer implementing
a business-to-business (B2B) infrastructure based on BEA Systems'
Web Logic middleware. "They're putting their own shell in because
although happy with BEA Systems now, they recognise over time they
may need to change their middleware supplier," says Hammer.
The shell handles all the calls into the underlying databases
and provides a consistent layer connecting these to the topware
applications. The middleware then plugs into this shell, which
provides the link to the back-end systems. It is then easier,
although still by no means trivial, to change the middleware,
because now it just involves re-implementing the shell without
changing the calls into backend systems.
Until recently enterprises needing to develop such a middleware
shell would have had to do the job themselves, but now tools are
emerging to help generate the interface layer. This layer needs to
comprise not just hooks, but also information about what is being
connected to, and what is being done. In effect, therefore, it
should comprise metadata, providing a map to the back-end databases
as well as information about how the front-end applications are
accessing this. The big weakness of existing middleware suppliers
is their lack of a metadata strategy, according to Hammer.
This applies even to the mature core application messaging
products such as IBM's MQ Series and Microsoft's Message Queue. The
latter is used mainly within the Microsoft environment, while MQ
Series dominates the enterprise messaging environment with 70% of
the market for software communication across platforms, as opposed
to within a single operating system environment. MQ Series is
almost synonymous with middleware in some minds, but while it has a
good reputation it does not by itself provide the data and process
management necessary to keep track of changes and ease the
integration burden as new applications are developed.
MQ Series allows messages to be routed within an application
framework to the correct system and then cause a process there to
execute, triggering as it is called. What it does not do is provide
a record of the routes and triggers, which would make it easier to
manage the environment and effect changes.
There are some tools now that help with this, one being ETI's
Extract Accelerator for MQ Series, which is a development aid for
generating the calls and providing an audit trail of all the links
to back-end systems and databases. Although currently only
available for MQ Series, such a product would ease the pain of
switching to another middleware supplier, because of the
information it provides on all the inter-process links.
Middleware though is increasing its scope beyond application
messaging. Products like MQ Series are asynchronous, providing a
loose coupling between processes, so that one might send a message
to another but not expect any response. There is also a need for
protocols that support direct interaction, integrating applications
more tightly together, and there are various products that allow
this, including some of the tools within the object oriented Common
Object Request Broker Architecture.
Because of the expanding scope of middleware, IBM has just gone
through a re-branding exercise, bringing all the products together
under the WebSphere logo. Not for the first time in IBM's case, the
re-branding exercise is more a statement of intent than a fait
accompli - ie, being a collection of often admirable but not
particularly compatible products. According to Geoff Rayner,
financial director in charge of IT at eHospital, an IBM middleware
site which specialises in B2B trading in healthcare, WebSphere is
not yet the answer to an enterprise's middleware integration
problems that IBM would like its customers to believe. "WebSphere
is an amalgamation of 1,000 things at present. People are confused
by what's in there, and we've been through that learning curve
ourselves just to find out what's in it."
Yet IBM's competitors can take little heart from these comments,
for Rayner reckoned that they are all less well prepared for the
challenges to come. "There's a big effort internally in IBM to
bring it all together. The good thing for us is we don't have to
pay for all that work."
In the hospital field, which Rayner describes as an IT nightmare
with its huge collection of incompatible systems from a wide
variety of suppliers, IBM's MQ Series is the best technology
available for providing at least a basic level of integration
between systems. This is providing real benefits for doctors and
other workers within healthcare, Rayner insists. For example MQ is
shortly to be used to integrate materials management with
procurement and the supply chain within some hospitals, which will
enable consultants, nurses and others to re-order surgical
consumables by scanning as they leave or enter a ward rather than
having to sit down in front of a terminal, or worse fill in a form.
The key to MQ Series' success, says Rayner, is the number of
different APIs it would communicate with, making it the best
technology to form the core of a middleware strategy for a
multi-supplier environment.
Not surprisingly, other suppliers dispute this relatively rosy
view of IBM's position, arguing that it ignores the expanding role
of middleware, beyond mere application integration, into unified
content and document management. According to Tom Bird, UK head of
collaborative working software supplier iManage, the missing piece
of many middleware solutions is the support for common access to
documents. "Over 80% of a corporation's knowledge exists in its
documents," notes Bird, who adds that middleware did nothing on its
own to open up documents and make them available to collaborative
working communities, both within organisations and in B2B
applications. The need is to elevate middleware into the spheres of
content, document and knowledge management, Bird asserts.
In fact this point has not been lost on traditional middleware
suppliers, and is one motive behind some of the recent rebranding
exercises. Indeed BEA Systems, which is the main challenger to IBM
in the middleware arena, is moving higher up the software food
chain, according to its senior architect for UK and northern
Europe, Mark Prichard. Particular targets for BEA, he says, are the
content management suppliers such as Broadvision and Vignette. "Our
differentiator compared with Broadvision is that we can provide
almost as much functionality, but based entirely on the Enterprise
Java Beans standards, with no proprietary kernel." This, says
Prichard, made it easier to integrate content management with other
key applications such as personalisation. "Enterprises prefer to go
with a single supplier, but also want an open standards-based
solution, even if the out-of-the-box functionality in a particular
vertical market such as content management is not quite as
deep."
With this comment, Prichard identifies the old dilemma for
enterprises, which is whether to go with a perfect set of point
solutions to individual problems, or compromise with a single
supplier framework that will have weaknesses in some areas but
eases the integration burden. In the case of middleware there is no
sign of this dilemma being resolved.
Key middleware players
Middleware is dominated by five big suppliers, in this case IBM,
BEA Systems, Oracle, Sun Microsystems, and Microsoft. According to
the Giga Group, IBM will take over the lead in the market for
multi-supplier middleware, with its share growing from 16% to 24%
during this year. BEA will be number two. Microsoft will remain a
big force for inter-application middleware within the Windows
environment with its COM and Active/X technologies, while Oracle is
strong among its own database customers. Both BEA and Sun have
their main base among Unix sites, with the latter more predominant
among users of its own hardware.
BEA is unique among the big five in being independent, or at
least perceived to be, from any particular hardware or software
platform. Yet it is IBM with MQ Series that is strongest in the
market for middleware linking Unix and non-Unix systems, via its MQ
Series messaging software.
Tips for changing middleware
- Understand why you are changing middleware, and work out what
is wanted, for example greater ability to integrate with other
systems you have
- Consider the change in the light of overall IT strategy. For
example you may be implementing mobile applications in which case
the new middleware must be capable of supporting pervasive
computing and Wap devices
- Assess the features required. Typical features include
asynchronous message queuing, message broking which facilitates
synchronous and more closely integrated processes, support for
distributed transactions, security, access to legacy systems, and
support for small mobile devices
- Look at the products. Many suppliers call their products
middleware, but with wide differences in functionality, capability
and all round coverage. A key point to consider is what level of
integration is needed
- Run a pilot project, especially if a major change of middleware
right across an enterprise is being considered
- Start by migrating just one application. If the new middleware
cannot cope, perhaps lacking sufficient support for key emerging
standards such as Compact XML for small devices, then it is less
costly to withdraw at this stage.
- Review the initial migration thoroughly before committing the
whole enterprise. Determine the roadmap for future platform
evolution, skills required, and define the enterprise's own
middleware architecture accordingly - do not rely entirely on the
supplier for this
- If all appears well, plan the full roll-out
- Hope that the project is not the next Computer Weekly
front page disaster story.