One of the main objectives of large-scale strategic outsourcing, of the kind seen between the Inland Revenue and EDS, is to provide value-for-money. But new research has found it difficult to prove.
The research shows that it is hard to measure whether multi-million-pound outsourcing deals can improve on smaller scale "tactical" outsourcing.
For the first time in any major state IT outsourcing contract, this issue has been explored in depth by an independent consultancy, Lorien, under contract to the Government's spending watchdog,the National Audit Office. The research found that it is not clear whether the Revenue could have achieved better value-for-money by steering clear of wholesale privatisation and transferring all 1,900 staff to one company.
By the standards of other major IT contracts, it has been a success. EDS has cut costs and improved productivity. It has also delivered new IT systems to support government policy, such as the tax self-assessment systems, without the major disruption that accompanied the introduction of new technology at the Passport Office and the Contributions Agency.
But Lorien suggested that the contract does not necessarily provide evidence that the privatisation of the Revenue's IT department has provided better value-for-money than a "more traditional adversarial approach to procurement in which there are frequent competitions to provide individual packages of well-defined services".
It said assurance of value-for-money could be obtained by two main methods: benchmarking and market-testing.
Benchmarking involved a comparison of EDS' prices, performance or a specific service, such as a call centre against other suppliers, and market-testing meant exposing proposed and existing services to competition. In the Revenue's case, Lorien found that both methods were flawed.
Now over halfway though the contract, only 37% of the annual spend with EDS was benchmarked (although more is planned) and market-testing was described by Lorien as "ineffective".
There were particular problems trying to compare the prices charged by EDS and other companies, said Lorien. The Revenue had "difficulty collecting price and cost information from other organisations". This was partly because other users or suppliers were unwilling to share sensitive information on prices with the Revenue, either because of commercial confidentiality clauses, or because they saw no advantage in doing so.
And EDS has faced little effective competition in the award of new work, partly because of the way the contract was set up.
The Revenue has to prove EDS' prices are out of step with the market. Then EDS has a chance to match the indicative market price. This, said Lorien, was likely to discourage third parties from providing proposals. Only if EDS declines to price match can a formal competition take place.
"So far, no existing service has been market-tested and one new requirement, the provision of an Internet capability, has been subject to open competition and let to another organisation," says the National Audit Office report .
And although the Revenue's contract provides for the cost of the transferred work to reduce year-by-year, Lorien found there was a risk that technological and services costs in the market generally might be reducing at a faster rate than at the Revenue.
Software development productivity had improved since EDS took over, said Lorien, but was still "generally below industry standards and could take a further five to 10 years to reach industry levels".
This was partly because EDS could not have completed the £200m project for tax self-assessment in the short time available if it had built entirely new systems. Therefore, the Revenue is still reliant on ICL mainframes that run VME-based systems dating back more than 10 years.Developing software in this technical environment cantakemuch longer than developing systems on the latest hardware and operating systems.
But some of the most significant of Lorien's findings lie in the difficulties that the Revenue and potential bidders will face when the contract is re-tendered in the next two or three years.
One of the reasons market-testing has been ineffective so far is that only certain EDS executives have a detailed and knowledge of the way IT systems integrate with taxation policies and the Revenue's business and end-user operations.
Lorien describes this as "client-specific" knowledge and says it could "disadvantage outside suppliers in an evaluation". This raises the question of whether any other supplier could hope to have any chance of convincing the Revenue that it could compete seriously with EDS in a re-tender of the contract.
The National Audit Office report on Lorien's findings is expected to be discussed at a meeting of the House of Commons all-party Public Accounts Committee next month.