Service oriented architectures are at the stage where "everyone is having a go, even if they do not have to". This was the verdict at IDC's SOA conference in London last month, where a large majority of delegates raised their hands when asked who was implementing SOA.
No one is fooled into thinking this sea of hands translates into universal prowess on the topic.
Delegates Computer Weekly spoke to came from a range of IT departments: from the blue chip company with four years' experience in rethinking and retuning applications to services, to higher education bodies and small businesses, lacking either the funding or an obvious business benefit to justify a pilot of an SOA.
However, listening to pioneers' stories some clear themes emerged. British Energy Power & Energy Trading turned to SOA to extract more business value from its IT investments, and EBS Building Society discovered that an "SOA-like" architecture improved its dialogue with business colleagues.
Many delegates said that adopting business process modelling was a prerequisite in order to share a vocabulary that would achieve SOA implementation and outcomes. Another theme to emerge was the need to start small now and not wait for a "Big Bang" implementation.
Network provider Vanco learned valuable lessons when it extended a service to an external partner. And its insistence on the importance of an upfront investment in design was echoed by many others.
Unlike many of the technical architectures that preceded it, SOA is less concerned with complex integration issues than with governance issues. "It is all about interfacing to the object and less about the detail inside," says Steve Elliott, SOA specialist at SunMicrosystems.
For this reason, issues about how you treat a service once you get to it are more perplexing than any connectivity concerns. The big questions, says Elliott, are, "How do I know if it is a trusted service? And how do I authenticate it?"
Likewise, IT professionals have to start realising that they are moving towards a federation where the services they produce and consume may be just a part of a composite answer.
Case study: Bepet
British Energy Power & Energy Trading (Bepet) totally rethought both the way it designs software and its relationships with suppliers as a result of shifting to a service-led model for delivering IT.
The energy company moved to a process-driven architecture, deliberately shunning the SOA label to disassociate itself from the hype. The moved enabled Bepet to prioritise processes and make IT better serve the business.
"Processes are the DNA of our organisation. We had to focus on higher-value activities, rather than factory-type programming, to be in good shape for future business. There are no prizes for second place," says Jeremy Lock, IT manager at Bepet.
Having an SOA would enable the IT department to support value added processes, rather than support functionality in a more piecemeal way.
In order to facilitate this shift, Bepet divided its business applications into three categories of services, defining a service as a self-contained and independent unit of work.
The three categories were: a task requiring a human decision, an information service and a functional service. Technology services support all of these three categories.
Converting these activities into services has exposed the energy company to some new ways of thinking about intellectual property rights and the execution of design.
"Traditionally we bought packages and did little bespoke development. We would lob our requirements into the market place, get bids back, build them, and then accept or reject," Lock says.
However, using an SOA entails a shift in thinking about intellectual property rights. Within the new regime, Bepet looks at the best of breed packages on the market, but does not customise. Where packages are lacking in functionality, the team writes a service to supplement it.
"Whether the service is inside or outside a package we have to connect to it. And the intellectual property rights of every service have to be captured within our model."
This means that Bepet may own the intellectual property rights of a service within a package, an unusual concept for some software and system integrators.
"The big five consultancies all have their own method for implementing packages. And we are now saying to them, 'we want you to do it our way'," says Lock.
Harvesting reuse, another major objective of the SOA investment, has also called for a radical rethink of design, says Lock. "You need to design business services at the right atomic level. And an upfront investment in design is crucial if you are going to get reuse later down the line. It really challenges all the normal paradigms of software design and support."
Even for companies like Bepet that are compliant with the IT Infrastructure Library (ITIL), breaking everything into more granular units makes everything more complex by default.
"We have a team of seven supporting 60-odd applications, and when I tell them we are breaking these into services, they are rightly concerned about the risks," says Lock.
Governance perhaps represents the biggest expenditure of effort in moving to SOA. "Only 30% of SOA is about development, the rest is about governance and managing services. Working with partners accelerates the adoption rate, but it is important to internalise the lessons and to assume control."
Case study: EBS Building Society
EBS Building Society, Ireland's largest mutual building society, arrived at an SOA model of delivery by accident.
"We have been doing integration and service work for years. We did not call it SOA until we discovered it written up recently and realised we were doing the same thing," says David Yeates, senior manager, IT architecture, at EBS.
EBS runs its mortgage origination systems as a portfolio of services. After all, says Yeates, "A mortgage is actually a portfolio of systems blended into a single product."
Over the years EBS had built general and life assurance systems, and had duplicated code and functionality as it went along.
"We realised we could break down this monolithic application and give some of it to partners. It is no different from what manufacturers have been doing for years."
The building society stumbled across a way of making this happen when it did an internal survey of reuse. "Our big moment came five or six years ago when we moved from simple, internal integration to rich integration. We had a proliferation of hardware and application servers - it was spaghetti."
However, EBS had difficulties in achieving reuse, even though it had been dabbling in object oriented programming. "Investigation revealed that the area where there was the most reuse was an ancient Cics application, which we had exposed to some terminals," says Yeates.
The building society also discovered that keeping an audit trail was a good model for delivering services, according to Yeates.
As a result of these discoveries, in 2001 EBS rolled out its Enterprise Application Integration Project, moving from a tactical to a strategic delivery. This encompassed building application and peer-to peer communication into its applications at the outset.
At some point during this evolution, Yeates and his team discovered they could talk to business users using this communication technology. A non-IT literate business user was able to track and discuss the development of a service by observing the flow charts and diagrams on the wall in the IT department.
"We were sharing the same vocabulary. But if we had talked about SOA, we would have been laughed out of the room," says Yeates.
During the experimental and somewhat accidental business of implementing SOA, data semantics was the biggest stumbling block, according to Yeates.
"The meaning of 'salary', 'name' or 'income' change according to who you are talking to. Nor do you have one system to deal with, but multiple ones, and that has huge implications for the IT department."
There are a number of other important lessons that Yeates can pass on, too. He is keen to point out that SOA is not the same as web services.
"We were doing SOA long before web services came along, using advanced program-to-program communication, remote procedure calls, remote method invocation and internet inter-orb protocol."
Nor is SOA good for everything. "We got carried away at one point. We use expert systems to generate answers to online questions, but realised exposing these as a service was not appropriate."
The biggest ongoing challenge for EBS is security. "If an external partner is providing part of your web service in real time they need to be working to a framework such as ITIL," says Yeates.
"Exposing your application in a business-to-business web service is a security headache that we are still finding our way around."
Case study: Vanco
Virtual network provider Vanco wanted to extend a trouble ticketing system as a shared business process to a business partner.
Vanco operates many such support services on behalf of its system integrators and channel partners on a sub-contract basis, and the company reasoned that making the technical support service visible would improve communication and create a better experience for the end-user.
Vanco deliberately started small: "We were not sure how many services we wanted to create and expose at this point," says Dave Doherty, global IT programme director for Vanco. His expectation was that the technology would be used to integrate other functions internally.
A small beginning was also pragmatic because it meant funding a shorter learning curve. "We run quite a small technology team and so wanted to limit the amount of new technology that we had to skill up on, even if this meant a trade-off of fewer features." The network provider selected Sun's Seebeyond software to meet these requirements.
Exposing a service to a third party threw up issues relating to the two-way interactions, and these had to be treated with great precision. Is it an outbound or inbound creation of a ticket? Whose responsibility is the fault handling? All these questions had to be nailed down precisely in order to create business rules at the centre, says Doherty.
Exposing a service to a third party also calls for greater programming discipline and version control. "There was one incident when we were halfway through an implementation and found our partner had upgraded some software. That put us back for half a day, for example."
The lesson Doherty learned is that although web services offer fantastic flexibility that enables you to connect to anywhere, the problem is that no web service is constructed exactly like another.
"None of them are vanilla flavoured," he says.
Case study: DKSH
Services, marketing and distribution company DKSH needed to connect to customers and suppliers faster in order to retain its pole position in a highly competitive market.
DKSH offers marketing and distribution for a portfolio of consumer and healthcare products and operates from four business divisions in 35 countries. The company employs 22,000 staff who manage 100,000 customers.
Given the scale of the operation, the major challenge was organisational. "Each country had a different culture and a different set of processes and systems to exchange information," says Alexander Buech, chief technology officer of E2E Technologies, DKSH's system integration supplier for the project.
And, because of past mergers, there were different ways of connecting to different local business partners, too.
DKSH initially tried a technically driven implementation that failed because it was not viable to separately specify technology and business requirements.
The company realised it needed a standard way of describing services that employees across business departments and across locations could understand.
The alternative was a model-based approach where the high level business logic became the IT delivery mechanism, avoiding error in the conversion of business abstracts into code.
Using a universal modelling language (UML) solved the problem because the model executes directly instead of generating code. It means no programming, and therefore a much faster turnaround for projects.
DKSH opted to use a E2E Bridge, a UML virtual machine from E2E Technologies as the enterprise-wide integration backbone, as well as three of the specialist supplier's programmers.
Over the space of 10 weeks the team modelled requirements, roles, services, systems, data and security created semantic mappings of metadata and associations defined service logic and specified the system landscape.
The resulting SAP template became the central hub of shared services with which every country's processes would integrate. This entailed persuading people to give up their attachment to proprietary systems and agreeing to exchange data in the selected SAP format.
"The new process instead consisted of taking orders in SAP, tracking in SAP, recording status and finally shipping the product to the customer," says Buech.
The approach enabled DKSH to achieve another key objective of no longer relying on individuals to correctly interpret the services between countries and reuse services.
In order to achieve this outcome, business intelligence employees were nominated as project managers and the IT departments became the service providers.
Although the business intelligence team were deemed the appropriate people to lead the project, they also had to be disabused of the "uniqueness" of their individual services. "They listed 78 services necessary to deliver all functionality - we whittled it down to 15," says Buech.
DKSH chose to pilot the SOA in Singapore. "The goal was to get rid of all incumbent enterprise resource planning systems and to replace them with a lean system for connecting manufacturers and resellers," says Buech.
With each subsequent country that came on-board, the reuse factor was higher and the roll-out therefore quicker.
Factors already existed that were working in favour of a successful outcome for SOA, says Buech.
"DKSH was a centralised operation and was ready to embrace the new way of working, because their future depended upon it."