Translating Research into Reality

My posting earlier this week on the costs of incidents created a few stirs, the most interesting one being an email from the excellent Ponemon Institute, who have been the source of many highly-publicised claims about the costs of data breaches.

Estimating the potential cost of security incidents is fundamental to corporate risk assessments and the resultant business cases for security spending. It’s clearly vital that security professionals have a sound model for estimating potential business damage. And the Ponemon Institute research is the most authoritative basis for this. Because it’s based on up-to-date analysis of real incidents. The Ponemon research also provides useful metrics for business cases, such as the total recovery cost per compromised customer account. Every security professional should become familiar with this research because it’s central to the justification for the resources and budgets needed to mitigate the risks of data compromise.

The difficulty of course is translating past research findings into future reality, especially when the scale is different. Such as in the recent incident at TJ Maxx, where many of us were tempted to extrapolate figures based on thousands of compromised accounts into estimates based on millions. And how well did we do that? Not at all well I’m afraid to say. Most analysts simply multiplied the historical average damage per account by the number of compromised customers. This projected a hit of several billion dollars prompting a wave of doom-laden warnings.

We should have listened to Larry Ponemon. Because he actually published statements at the time pegging the projected cost in a range of “hundreds of millions” of dollars. The TJX Group initially claimed a total cost estimate of around $25 million, but recent updates have inflated that figure by ten times, square within the range projected by Ponemon. What the pundits overlooked was the fact that TJ Maxx was an exceptional case. The breaches studied by Ponemon were in the range of a few thousand to a quarter of a million. The TJ Maxx incident involved more than 45 million cases. But, as Ponemon point out, many of the costs associated with data breaches are not fixed ones. The larger the breach, the smaller the resulting per-record number.

So well done Larry for getting the projection right. And there’s a clear lesson for us analysts and pundits to be a little bit more cautious in translating research into reality.