Data Integrity - The Final Frontier

Regular readers of this blog will know that I’ve been forecasting for some time that data integrity will be the next big thing. That’s nothing new. But what’s really interesting is that many of my fellow security professionals are now starting to say the same thing. Data integrity was certainly one of the hottest issues raised at last week’s Infosecurity Europe Advisory Panel. I’ve previously commented that it might take five years for people to respond to this challenge. Hopefully, awareness of the problem space might start to take off during 2010.

Data integrity is the third and arguably the most significant phase of information security. It’s the final frontier to be tackled in contemporary information security, which is based on the three pillars of confidentiality, integrity and availability: a long-standing fusion of three distinct objectives that collectively map out a solution space that still contains many gaps. It’s understandable that people tend to notice the availability and confidentiality aspects of security well before they spot the integrity issue. But the integrity challenge is quietly building up into a dangerous exposure. Bad data undermines business confidence, and in extreme cases it can permanently reduce the value of business services.

So why is data integrity such an issue? Firstly, much of our data is already bad but we don’t advertise that fact. We keep it quiet. In many databases, it’s not unusual to find that up to half the records contain errors of one sort or another. That’s due to a combination of factors, ranging from transcription errors in call centres to the inevitable temptation to re-use old data outside of its original context. On top of that we have a range of network effects that distort incoming data through Chinese whispers, rumour, spin or good old fear, uncertainty and doubt. There’s a tendency to believe anything that you hear from several different sources. In large networks, that can be deadly. But the most disturbing concern is the threat of an unauthorised intruder deliberately changing data to cause harm, whether for financial gain, spite or sabotage.

The starting point in addressing this relatively new problem space is to recognise that we need standards to assure customers, citizens and other stakeholders of the quality of the information in our databases. It’s quite outrageous that none exist for services that can have a major impact on people’s lives. A single percentage of error in a national database can represent a population the size of a major city. That demands scrutiny. Once we can see the size of the current exposure, there’s no doubt that society and the media will demand action. But until that happens we’re sitting on a ticking time-bomb that’s just waiting to explode.

Join the conversation


Send me notifications when other members comment.

Please create a username to comment.

David is absolutely right that integrity is seen as the poor sibling of confidentiality and availability, but this has to change. Bad data integrity is not just bad for business but processing inaccurate or incorrect personal data is also a breach of the Data Protection Act. The fourth data protection principle states that Personal data shall be accurate and, where necessary, kept up to date. With the increased focus on Data Protection and the Information Commissioner’s new powers, we’re going to have to take data integrity a lot more seriously.
Hi David, Thanks for an interesting blog. Integrity is perhaps the least well understood and least well addressed aspect of information security. There is another side to this issue as well...the obverse of integrity is authenticity...a term with very significant meaning in the legal field. I posted today a POV on infoBOOM! ( which is intended to get people thinking more about the potentially underaddressed issues of data integrity and authenticity. I'd appreciate knowing what you and your readers think about it. --Paul
David, Another consideration besides integrity of the data, is importance of the integrity of the audit logs, for forensic defensibity in matters where things like proof of due diligence is required (compliance, governance, prosecution). For this reason, immutable audit logs are highly desirable also. Immutable audit logs are a great deterrent against unauthorized tampering with the the data.