Information Security is a maturing discipline, writes Lee Newcombe, principal consultant at Capgemini. In the past it was possible for canny security managers to obtain funding for security projects and countermeasures through judicious use of fear, uncertainty and doubt. These days we are expected to use more conventional means, such as robust business cases, to justify investment in security controls.
It is necessary for those responsible for information risk in an organisation to judge the effectiveness of their strategy and associated security solutions. Many would accept, and are attempting to implement, metrics programmes to measure the business value and/or on-going effectiveness of an information security regime. However, metrics programmes are not without their own pitfalls.
There are some fundamental questions that need answers before using a metrics programme. Firstly, what is the purpose of the metrics programme and who are the main beneficiaries? The answers should help to shape the answer to another question - which metrics should the programme monitor and report upon? There is also the question that perhaps does not get asked as often as it should - does the cost of initiating, implementing and maintaining a metrics programme outweigh the benefits the programme would provide?
The answers depend on the organisation concerned. Metrics should be designed around the risks and objectives of a specific organisation. Monitoring the wrong metrics can have serious consequences, particularly as managers will have the full backing of the new metrics programme to justify their (potentially ill-advised) decision making.
Another issue that can adversely affect a metrics programme relates to that stubborn cause of many security misfortunes - the human factor. If a metric is defined that measures the number of reported security breaches in specific business areas, what is the motivation for an owner of a business area to report breaches? Should a business owner be rewarded for reporting breaches (and perhaps encourage more breaches), or punished for suffering the breaches in the first place? In the latter case the individual will almost certainly become more reluctant to report security issues - definitely not the desired outcome!
Some of these risks can be mitigated through a well-planned exercise to derive the initial metrics based on an analysis of the business objectives of the organisation, the security policy and culture of the organisation and the results of a comprehensive risk analysis. Metrics can be chosen that measure trends in key risks that directly affect the business. Metrics should be tested in pilot environments before a wider roll-out in order to check they can be economically obtained and provide the expected benefits. This pilot exercise can also be useful to check the metrics programme itself is not inducing unintended behaviours in those covered by the programme.
Metrics programmes can be expensive and result in reports and dashboards that are unused or worse, meaningless. However, if designed and implemented correctly, a metrics programme is a great way to demonstrate to business stakeholders that risk is being managed appropriately and to provide justification when investment is needed in additional security controls.
About security zone
Security Zone is a bi-weekly series in Computer Weekly covering all aspects of IT security management. Each article will be written by a member of the International Information Systems Security Certification Consortium (ISC)2.
This was first published in September 2008