Stuart King's Security and Risk Management Blog

I must admit that I am still struggling with metrics. One problem is now between what I believe are useful statistics to be gathering against what's useful information for management. Another is the whole gathering exercise to consider: especially as I'm asking individuals to spend more of their valuable time reporting data back to me at the same time as they are receiving umpteen other requests for data from various other sources ranging from Corporate Audit to the Project Office.

I've reviewed what the purpose of metrics are because the danger is that we'll just set about gathering data for the sake of it. The objectives, in my opinion, are to

a) Collate information about the security status of the organisation
b) Measure progress (i.e. to answer the question "Are we more secure today than we were yesterday?")
c) To assess the impact of changes

We also need to be careful not to utilise metrics as a competition between different parts of the business. The motivation should be to understand impacts and risk, and the role of the security manager/director/officer (let's talk about job titles sometime shall we?) to provide management, guidance, and resources for mitigating risk. What we don't want is more league tables because then we'll be ring-fenced and have to work much harder to get the data we want.

I took the following quote from a paper by Shirley Payne entitled A Guide to Security Metrics

Good metrics are those that are SMART, i.e. specific, measurable, attainable, repeatable, and time-dependent, according to George Jelen of the International Systems Security Engineering Association. Truly useful metrics indicate the degree to which security goals, such as data confidentiality, are being met, and they drive actions taken to improve an organization’s overall security program.

This blog from Gary Hinson contains some references to various useful materials, including his interesting paper entitled "Seven Myths About Information Security Metrics." Another good reference is David Lacey's paper entitled "Top 10 Tangible Measures for Effective Security Risk Management" linked from his blog here.

So, there's plenty of guidance out there but much more interesting would be some actual comparison of what's being used by different organisations in the real world. Once I've finished defining the format of what I'm planning to collect and collate then I'll share it here.