SATs and government praise of ETS

When the government awarded the contract to ETS to process and mark SATs, Whitehall’s press release was, as to be expected, full of praise for the supplier and the procurement process.

But the press release showed how the government, in awarding multimillion contracts, can be not merely at arm’s length but miles away.

The press release in 2007 announced a five-year, £156m contract to ETS. It was written by the National Assessment Agency, which is part of the Qualifications and Curriculum Authority which is part of the Department for Children, Schools and Families, which is a successor to the Department for Education and Skills.

The press release quoted David Gee managing director of the National Assessment Agency:

“Today marks the culmination of a thorough and exacting procurement process. We are confident that, together with ETS, the National Assessment Agency can continue to deliver a robust and efficient National Curriculum assessment system fit for the 21st century.”

Kurt Landgraf, ETS President and CEO of the parent company, ETS, said:

“… As a global, non-profit company, we are dedicated to assuring quality and equity for all students.”

Now that the partnership has gone sour the government is using as its defence its arm’s length arrangements. Schools minister Ed Balls said on 21 July 2008:

“The contract with ETS Europe was drawn up and has been managed by [the] Qualifications and Curriculum Authority, at arms length from Ministers, to ensure the independence and objectivity of the testing regime.

“Any discussions about the contract are legally a contractual matter for the QCA and ETS Europe…the independent inquiry will look at all the issues surrounding the test delays, including the specification and procurement of the contract with ETS.”


The inquiry should look at whether ministers should have their own independent sources of information on the progress or otherwise of major IT projects.

Ministers are not allowed to see the results of independent gateway reviews on a project in their department unless the senior responsible owner decides to disclose them.

This secrecy – which the Information Commissioner regards as excessive – is because the Office of Government Commerce has decreed that the results of gateway reviews are confidential unless the SRO decides to share them. All copies of gateway reviews and supporting material are shredded, leaving only one copy for the OGC which is kept in a Swiss bank, or electronic equivalent. The other is for the SRO to share or not.

Since ministers are routinely excluded from seeing gateway reviews, who can provide the challenge to assumptions and independent scrutiny of major projects? Not even suppliers are shown the results of gateway reviews as a matter of course .

It’s not clear whether gateway reviews were carried out on ETS’s systems. Perhaps not. It was, after all, an arm’s length arrangement.


Ministers knew little or nothing about the seriousness of the impending problems before the failure of the Single Payment Scheme at the Rural Payments Agency. The scheme was hit by a “conspiracy of optimism”. Ministers were given repeated, positive assurances that the risks were being successfully mitigated. Similar assurances were given before the tax credits problems.

In 2007 Tony Blair was given warm assurances about progress on NPfIT, the NHS’s National Programme for IT.

Isn’t it time ministers and MPs – and in this case parents and children – ceased to be victims of Whitehall’s obsession with secrecy and particularly suppressing bad news from which lessons ought to be learned and usually aren’t?

Ed Balls’ [edited] letter to the Children, Schools and Families Select Committee on 21 July follows:


Dear Barry

Further to last week’s Select Committee hearing, at which I announced the terms of reference of the Sutherland inquiry, I am writing to update you further on developments in relation to National Curriculum testing and the delivery of 2008 Key Stage Two and Three tests.

As I said when I appeared before your committee last week, our first priority has been to ensure that schools receive their 2008 results in an orderly way with the minimum of delay. I am advised that 98 per cent of Key Stage Two results are now available to schools online, as well as 85 per cent of Key Stage Three results – 93 per cent of maths results, 91 per cent of science results and 71 per cent of English results.

I share the frustration and anger of teachers, children and parents about the delays in the release of test results. Whilst the large majority of test papers have now been marked and results released to schools, I am concerned that schools should get outstanding results as soon as possible.

In a statement issued over the weekend, the QCA have confirmed that they are in discussions with ETS Europe following the unacceptable delays in delivering this year’s National Curriculum test results. In that statement, the QCA say that it is considering all available options to allow the timely conclusion of the work for the 2008 test series, and to secure a successful 2009 programme. I attach a copy of the QCA statement.

The contract with ETS Europe was drawn up and has been managed by QCA, at arms length from Ministers, to ensure the independence and objectivity of the testing regime. Any discussions about the contract are legally a contractual matter for the QCA and ETS Europe. At this stage, I believe it is very important and in the public interest that the QCA should be able to conclude these discussions in a timely and orderly fashion in order to safeguard the interests of pupils, schools and taxpayers.

At the same time, Ofqual, the independent regulator of qualifications, exams and tests in England, have assured me that they continue to monitor the quality of the marking of the tests, will consider the evidence in relation to any problems that are brought to its attention, and will act accordingly.

I am determined that we learn the lessons of this year’s experience, and I am confident that Lord Sutherland’s independent inquiry will be of great value in this regard. The independent inquiry will look at all the issues surrounding the test delays, including the specification and procurement of the contract with ETS.

Both the work of Ofqual and the Sutherland inquiry are designed to secure public confidence that the system is being appropriately regulated and scrutinised at arms length from the government, and I hope they will be useful to the Committee’s own activities…




Thank you to the South West Somerset branch of Unison for drawing my attention to the National Assessment Agency’s 2007 press release.  Unision has many concerns about a £400m outsourcing deal which involves Somerset County Council, Taunton Deane District Council and Avon & Somerset Police. It makes the point that a gushing press release on the signing of a large contract sometimes precedes a fall.  


Conspiracy of optimism hit rural payments scheme – Computer Weekly – 2006

Civil servants told to destroy internal reports on billions of pounds of risky IT projects – June 2007, Computer Weekly

High Court case to keep gateway reviews secret – IT Projects blog, March 2008

Ed Balls, Schools Secretary, refuses to apologise over SATs fiasco – Daily Telegraph 

Sats exam fiasco: Ed Balls admits much marking is still to be done – The Guardian

SATs tests – debate in House of Commons – July 2008

Some NPfIT “major issues” – did the PM get a full briefing? – 2007 

Join the conversation

1 comment

Send me notifications when other members comment.

Please create a username to comment.

In an emailed letter to The Times, a correspondent says: "What I have not seen or heard reported is that ETS conducted a pilot study using their systems in the months up to Christmas 2007. I took part in this and it was, if anything, worse than the recent problems."

So despite the public statement that have been made by senior officers in the NAA and the QCA, there was evidence at the beginning of 2008 that serious problems existed regarding the operation of the new ETS systems and processes.

This is the sort of thing that the Gateway 4 (Readiness for Service) Review is supposed to pick up, to prevent the repeat of previous service delivery fiascoes. So the obvious question is what lessons did QCA learn from this pilot study and why were Ministers not made aware of the impending disaster.

It is ridiculous that public money is being wasted on poor quality service, when the contracting authority had evidence of the impending failure, and that the knowledge of the relevant organisations is hidden behind OGC secrecy which protects the SRO.