Measuring Security Progress in an Uncertain World

One feature of the Information Age that I find fascinating is how fast we are able to adjust to wild swings in levels of activity. We take huge falls in stock market capitalization levels in our stride, though they impoverish or enrich many citizens. We live with rapid changes in fashion – the Gartner Group hype cycle for example being a classic illustration of anticipated volatility in market perception. And it’s the same with security incidents. For years they’ve been increasingly volatile yet somehow we’ve manage to contain the risks. And over the last year we’ve seen some extreme behavior in malware. It’s really booming, growing by a reported factor of five during 2007. Now that’s serious growth.

Of course we can explain the growth by drawing attention to factors such as the fact that criminals have taken to pumping out multiple variants of viruses to fool anti-malware systems. But we can’t easily anticipate such trends. And they must be disruptive for academics aiming to develop better ROI calculations based on measured incidents. Or for that matter for any security manager who has been unwise enough to agree a bonus objective related to a reduction in incoming malware.

The simple reality is that volatility and uncertainty are the names of the game when it comes to information security risks. Low level threats can unexpectedly scale to new heights quicker than you can revise security defences and management objectives. Security managers need to recognise this level of uncertainty and deploy countermeasures that can scale well beyond normal contingency levels. And we need to work to realistic objectives that are not hostages to fortune. As Deming pointed out many years ago, it’s a deadly sin to manage on the basis of visible information alone. It’s the underlying trends that really count. You need to explore well below the tip of the iceberg to see what’s really happening.