“The still calm voice that drives the strongest of men to panic”. Today the Audit Commission launched a discussion paper “Nothing but the Truth” to start “a discussion on how to ensure that data on local public services is fit for purpose”. Read it. Think. Then be afraid. Because some of the data on the files of central government is much worse.
The paper raises profound issues regarding the quality of information used by Central and Local government for policy formation and resource allocation, let alone decisions affecting the lives, livelihoods, health, welfare and freedom of individuals.
The reasons why the base data is so bad include widespread and long-standing ignorance of the basic disciplines of information management not only across public and private sector but also among those selling “solutions” to them.
The disciplines appear to have been lost to sight sometime in the 1980s when Information Technology replaced Data Processing as the fashionable terminology and the study of sophisticated software replaced the study of people processes in the professional education of those entrants to the Information Systems industry who received any. Far too many of those who pass for “professionals” today have received only a hotch-potch of product or methodology focussed short courses and on-the-job training.
When I did my basic systems analysis training in the late 1960s we were taught to assume random errors rates of up to 10% on original data entry unless the material was entered and checked by those with a vested interest in its accuracy and with the knowledge and authority to ensure that errors were identified and corrected. We were also told to assume that it would subsequently degrade at about 10% per annum unless actively used and updated by those with the knowledge and ability to update the files.
Data collected for statistical returns or performance measurement was to be assumed to be random unless produced as a by product of an operational system. That assumption was confirmed for me when I saw how the “government returns” were actually made: usually same as last month with a random fudge factor, unless the person responsible had time on their hands and nothing better to do. I also saw the whole of one firm’s exports for three years go through in one month when a colleague did indeed have time on his hands and got round to reading his job description. Despite the bluster over false returns, HMG could not resist the ability to have a headline in the papers about the balance of payments turning the corner.
When I looked at Local Authority rating systems and utility billing systems for the newly formed Regional Water Authorities in the mid 1970s I learned that the rate of change among those to be billed for the services delivered to properties ranged from under 2.5% in the leafy suburbs to over 400% (i.e. an average length of tenancy of under 3 months) in inner cities. Hence the reason for coin in the slot meters for gas and electricity for the landlord to collect. The churn today is no less – and may be even more. Hence the value and limitation of residents’ registers and cards around the rst of the world. Hence also the widespread professional disbelief that any UK national identity system would be of any more value – “not even a lead standard” – and certainly not worth billions of new money.
Meanwhile there is a widespread view among Intenrt enthusiasts that if you mash-up garbage from a variety of sources and feed it through sophisticated software you get something other than digitised slurry.
Do read read the Audit Commission discussion paper carefully. Pick up on the snippets of detail. Follow up the references. Despite the problems, some health authorities, for example, have patient records that are remarkably accurate, because they are based on the data entered and validated by the clinicians who use it for patient care. Others record and report data that is actively fraudulent, manufactured to demonstrate progress against targets with little or no relation to underlying reality.
But, more commonly, staff under pressure to service a client will manufacture entries to get the system to do what is needed there and then. Thus some-one with poor english, no fixed address and no proof of indentity may be added several times over, under different names, even without any fraudulent intent.
The systems they use have been designed by people with little or no understanding of the needs of pressures at the point of service delivery, let alone the disciplines of information management. They are not fit for purpose. And there are far too many of them.
The Audit Commission have asked for comments on the discussion paper. Do respond.
The Information Society Alliance, EURIM plans to work with the Audit Commission to help take debate forward and identity consensus on how to address the problems raised.
Next week the Alliance is due to release a short working group “status report” on how treating information as an “asset” can be used to change the Information Governance culture and cause organisations to invest in quality improvement not just liability reduction. I will blog on this when it is released.
Please email email@example.com if you would be interested in helping the follow up, indicating the contribution that you or your organisation might wish to make to any follow up – including to turn consensus into action.