This is the third article in our series on next-generation enterprise IT, produced in association with IBM. There is also an accompanying podcast on BPM and SOA for business agility.
Translating business needs into software outcomes has long been an Achilles heel for the IT industry. Whether the approach has been to write bespoke applications or find a best-fit package, the result is normally an imperfect reflection of what was required. More seriously, the resulting systems are a static reflection of the needs of a particular moment in time. The weakness of this approach comes when a business needs to radically alter the way it operates, whether in response to changing market conditions or simply a desire to reinvent itself. When the systems that supposedly support the business can't be changed in a timely or cost-effective enough way to make that change happen, the business finds itself set in electronic concrete.
Many businesses have adopted the tools of business process management (BPM) to get a handle on how their business operates and to automate many key processes: the ease with which this allows businesses to streamline or upgrade processes only brings into focus the inability of systems to respond. This is where service oriented architecture (SOA) comes in: creating IT assets in such a way that they can flexibly respond to changing business needs.
"BPM is the 'what', controlling whatever the business process is designed to do," explains Gary Gomersall, IBM's SOA evangelist. "SOA is the meat of how you drive towards that business outcome."
The term evangelist is not hyperbole. IBM has just completed a 10-day, 100-city world tour to spread the word about SOA, a sign of its determination to push it as an approach that offers real business benefits. And the key to unlock those benefits is BPM.
"The whole basis of the link between BPM and SOA is the need to focus on specific business outcomes," says Gomersall, adding that the benefits of SOA can be grouped for this purpose into five areas: business flexibility, better business practices, easier integration, re-use of assets and reduction of risk. For Gomersall the combination of the two approaches gets us nearer the Holy Grail of IT in business: the ability to have real business policy statements translate automatically into outcomes delivered by IT systems.
"Invoking BPM though SOA means you can externalise the ability to set business policies and rules at business level language," he says.
Where business processes touch systems they can be thought of as invoking a service: where the systems are actually constructed as services it allows those services to be invoked by many different processes.
This has clear implications for business flexibility, but this doesn't mean that every IT system has to be rewritten as a cluster of services.
"You can turn every IT operation into a service but it's probably not valuable," says Gomersall. "Find out what business outcomes you need: easier integration, for instance."
IBM calls this approach Smart SOA and has defined a number of entry points from which organisations can begin to build an SOA strategy - connectivity or the desire to reuse assets, for example.
The problem with SOA is that an awful lot of the IT installed base simply isn't built that way: integration efforts that look good at the business level simply get bogged down in systems which can share data but little else.
"Early enterprise application integration (EAI) products had the capacity to link at the business process level but because there were no standards they never got beyond the data," says David Nichols, Accenture's lead for SOA.
"In ERP implementations you get bogged down in the 'system of record': you had to log on to a system to access a business process and every system had that locked down."
Service orientation was - and to a certain extent still is - in danger of being relegated to a way of integrating systems.
"A lot of people were forced into that," says Nichols. "But now we can move further up the stack and integrate at the business process level."
Service oriented approaches have gained a boost from two very different directions: the need of organisations to build highly customised applications and conversely the rise of "through the wall" vendors who sell software-as-a-service".
"Custom applications are wide open: if you're going to build an application you want to build it with the best services so that it's easier to maintain, easier to change," says Nichols. "While software-as-service products such as Workday and Salesorce.com are designed to accommodate a particular business purpose but are flexible enough that you can use parts of them for other purposes."
This leaves traditional ERP vendors with a conundrum:
"Do they go along with SOA or resist it?" asks Nichols. "If they go along with it they risk giving up their 'secret sauce', or they can resist and find the boat has left without them? Before SAO and Oracle there were other products that didn't survive because they couldn't move with the market. They're taking very careful steps but even the major ERP vendors are re-architecting their products."
Nichols says that underpinning BPM with SOA is gaining a lot of mileage with government departments.
"A lot of government agencies are buying into this because they know that their business processes will have to change - administrations will change, demographics will change, the economy will change," he says. "They know they may have to employ new things to accommodate these changes, therefore they want to build something that is much easier to change."
BPM/SOA offers clear advantages to businesses needing to adapt their business processes or create new ones. Rourke McNamara, director of product marketing, Tibco Software, estimates that one client - a European telecommunications company - was able to add broadband to its offering much faster through this approach.
"They'd already used the SOA approach in their existing fixed-line and mobile businesses, and decided to adopt a full BPM approach to their next business," he says. "They were able to build it 60% faster trough the ease of integration and they were roughly twice as fast to market in provisioning new customers by using BPM on top of SOA."
But, McNamara says, the company also gained the ability to adjust its business processes to deal with unusual events, such as the sudden availability of extra service personnel.
The addition of "complex event processing" (CEP) technologies to the BPM SOA mix takes the concept of business agility to new levels:
"What we try and do is take the things that happen intuitively and automatically for the best 10% of their employees and make the actions of those folk part of the underlying logic of the system," says McNamara.
This is many ways a reversal of the previous logic of process automation:
"In the early evolution of BPM we took the things that were 'brain-dead" and automated them so that people wouldn't have to do them," says Rourke. "Then when we wanted to ensure customers had a consistent experience we in effect dumbed them down, everyone had to operate at the level of the average. Now we're taking the intelligence of the best people we have and building it into the process."
McNamara cites the example of Con-Way, a truck company in the US: "The best truckers would monitor the weather and the traffic and change their routes but not everybody did that," he says. "Now they use the system to send advice to truck drivers on weather and traffic."
Not only that, but the system will also monitor when a package, despite everything, is not going to reach its destination on time and alert an agent to contact the customer.
This modelling of an alert employee has been employed by gaming chain Harris Casinos in an attempt to recreate the days when it was a single casino whose staff knew all the customers personally.
"Through their loyalty card they know when each customer is at their limit, the point where they won't lose any more money," says McNamara. "At that point the system investigates what free inventory the casino can offer - seats at shows for instance - and will send out an agent to talk to that customer and offer them free tickets."
This extremely dynamic use of BPM and SOA is rare, but McNamara sees it becoming much more widely adopted - in airlines for example, which he describes drily as "an event-rich environment".
The future of SOA seems to be in the balance at the moment - IBM's tour clearly recognises that it needs grass roots support more than vendor push, and will disturb some deeply entrenched hierarchies. As Accenture's Nichols notes, it can even be seen as a threat to companies like his own. Ultimately, he says, BPM and SOA need each other to be successful, otherwise they will be relegated just as enterprise integration was.
"And that would be a shame," he says.