The practice of sending across the country unencrypted, CD-based files on millions of child benefit claimants could have continued indefinitely if the discs hadn't gone missing, we have learned.
Seven months before the CDs went missing, HM Revenue and Customs had already established a practice of transferring onto CD, for despatch by post, insecure, though password-protected, files on millions of child benefit claimants.
The lost discs contained details of all child benefit recipients: records for 25 million individuals and more than seven million families.
The records included parental names, addresses, dates of birth, child benefit and national insurance numbers and where relevant bank or building society details. Paul Gray, the chairman of HM Revenue and Customs, has resigned because of the incident.
The practice of transferring all of the child benefit data onto CDs began in March this year after HMRC's auditor, the National Audit Office (NAO), ceased to accept sample records for its audit of the department's accounts.
In the past officials at the Department for Work and Pensions had selected sample child benefit files and passed these to the NAO whose auditors checked for possible fraud and error.
But in March this year, for an audit of HM Revenue and Customs's 2006/7 Resource Accounts, the NAO, to do a more robustly independent check on the child benefit data, requested a full copy of the details of claimants, not merely a part of the data that had been selected by the department.
Though HMRC does have rules on handling sensitive data, it is unclear whether it had specific, established procedures for handling the request of the National Audit Office.
Aware that the files on child benefit claimants were sensitive, the NAO in March 2007 asked that HMRC filter the information before sending it to the audit office. The National Audit Office asked for the child benefit records to be stripped of details of the parents, addresses and bank information.
HM Revenue and Customs replied that it could not do this - its systems were not sufficiently flexible. It explained it could download only the whole of the information. So it sent to the NAO, by courier-post, all of the details of parents and children, including some bank account details.
That was when the insecure practice began of HMRC sending unencrypted files to the National Audit Office. No alarm bells were raised over the practice in March 2007.
It appears that it was thought easier to send the claimant files on CD than trying to send them electronically. This raises questions about whether government departments are routinely sending CDs with sensitive data around the country, thus avoiding technical challenges and security restrictions on exchanging files electronically.
So in March 2007 HM Revenue and Customs transferred the child benefit data onto CDs and sent them by courier-post from Washington, Tyne and Wear, to the National Audit Office which is near Victoria Station in London. They arrived safely - and the practice became established.
The data was sent to the NAO only partially formatted. It had to be loaded on the National Audit Office's mainframe systems before it could be manipulated.
In October this year, when the NAO wanted to do an audit of HMRC's 2007/08 Resource Accounts, it again asked the department for its child benefit data.
The sequence of events:
2 October 2007: The NAO formally asks HM Revenue and Customs for files on child benefit claimants.
18 October: HMRC tells the NAO that the CDs have been sent
24 October: The NAO informs HMRC that the discs have not arrived. The NAO asks for a second set to be sent - it needs them urgently to ensure an audit of HMRC's accounts is not delayed.
25 October: The NAO confirms receipt of the second set of discs. It staff point out that the first set has still not arrived.
5 November: HM Revenue and Customs confirms that the first set of CDs is still missing.
8 November: The NAO begins a search for the missing CDs and the loss of the data is raised formally as a security incident. It is only at this point that HMRC's senior management is informed - but not the Chancellor of the Exchequer Alistair Darling who is responsible for HMRC.
10 November: HMRC with the cooperation of the NAO begins a search for the CDs at the offices of the audit office at Victoria. The NAO has no record of having received the first set of CDs. Only now is Alistair Darling, the chancellor, informed.
11 November: HM Revenue and Customs and the police search the NAO's offices. Nothing is found.
20 November: Alistair Darling makes a statement to the House of Commons on the missing discs and Paul Gray, the chairman of HMRC resigns.
21 November: HM Revenue and Customs issues an apology.