Are OGC gateway reviews working?

A reader has written a trenchant response to an article on this blog about the failure of a project to manage the registration of markers of school exam papers, and the processing of marks into grades.

The reader’s email is a reminder that the Office of Government Commerce’s gateway review scheme was launched with the object of pre-empting high-profile IT-related disasters, particularly in central government.

It’s likely the scheme has been successful in guaranteeing the success of some smaller and medium-sized projects and programmes. But there’s been no cessation of major failures since the OGC began doing gateway reviews on risky IT and other projects.

The OGC says gateway reviews have saved hundreds of millions of pounds. When asked whether the figures for savings have been audited by the National Audit Office its spokespeople reply that the gains have been calculated using a methodology agreed by the National Audit Office. In other words “no”.  

It turns out that the savings estimates are based on extrapolations – not a great deal more scientific than hopeful guesses.

The secretive, cosy world of gateway reviews limits their chances of making a real difference. The reviewers are not held accountable for their audits and decisions. The Senior Responsible Owner of a project is not accountable for taking too little notice of reviews. The OGC’s complacent, comfy rules over the conduct of gateway reviews mean that nobody need know what a gateway review says, apart from the senior responsible owner.

Failure after intelligent perseverance and learning lessons is one thing. Failure after repeating mistakes of the past is not easy to understand. And we’ve seen a repetition of past mistakes in the way the NHS’s National Programme for IT was launched, the way the Rural Payments Scheme failed, and the way failure over the scheme to manage the marking and grading of school exams came only slowly to the consciousness of ministers and senior civil servants.

It’s against this background that a reader writes (in response to our article about BBC’s Ten O’Clock News):

“So here we have yet another failure for the OGC’s Gateway Review process. Given this was a national programme, with a high impact on schools and pupils, One wonders what the outcome of the Gateway 4 (Readiness for Service) Review was.

“This has considerable parallels with the New Tax Credits (NTC) rollout – a brand new process, requiring registration – as the BBC says “Hundreds of markers could not register in time causing long delays” which is similar to hundred of taxpayers not registering as early as the project team expected for NTC.

“Again despite published advice on major IT projects this appears to have been a “big bang” implementation – all SATs results for 2008 being processed through the brand new system.

“So did this project have a realistic implementation plan, what were the plans to manage the registration process and monitor ‘take-up’ against plan, were the delays in marker registration a consequence of late system availability or initial poor system performance?”


BBC TV reports on “another” government IT failure – IT Projects blog 

SATS test delays inquiry is launched – BBC

High Court case on ID Cards gateway reviews, the 1689 Bill of Rights and the Speaker of the House of Commons – IT Projects blog

Bad news for transparency? – Spy blog

Do gateway reviews produce results? – Computer Weekly, 2007 

Join the conversation

1 comment

Send me notifications when other members comment.

Please create a username to comment.

I just spotted your blog and your reader's response. I am not at all surprised at the SAT tests fiasco.

OGC's Gateway review process doesn't deal adequately with the fundamental issue behind most of these failed or delayed projects: the high complexity of the project relative to the low capability of the organisation to manage its implementation.

After spending years as a change management consultant being frustrated at seeing so many modernisation and efficiency programmes failing to deliver the expected benefits, I started looking at the literature for some of the underlying causes - and found that I was not the only one to notice this problem, and its not just in the public sector!

- A recent CIPD report, Reorganising for success: CEOs and HR Managers’ perceptions surveyed 800 executives and found that reorganisations failed to deliver real improvement in performance in 40% of cases.

- Standish Group surveys confirm that in large companies, only 10% of IT projects are completed on-time and on-budget

- The Harvard Business School tracked the impact of change efforts among the Fortune 100 and found that only 30% produced a positive bottom-line improvement.

But the problem does seem to be greater in the public sector - why? I encountered all the usual reasons in my research, but felt that these didn't really explain the high level of failure.

In the end I concluded that initiating organisational changes of any kind presents both a process management challenge and a people management challenge to the organisation. If the complexity of the project is such that it overwhelms the organisation's capability to manage it, the consequence is, at the very least, a project that costs much more, takes far longer and delivers considerably less than planned.

I am now using tools to measure project complexity against organisational capability, and coming up with some interesting answers. In the latest SAT tests case, I am reasonably sure that ETS Europe failed completely to understand the complexity of the project it took on, in the context of the culture and capability of the UK education sector. The tragedy is: so did the National Assessment Agency.