I thought it would be interesting to look more closely at the TCO (Total Cost of Ownership) and ROI (Return on Investment) associated with an IDS/IPS and see whether or not we can realistically make a judgement on the investment using these as metrics. If we can’t do so then there’s the possiblilty that any supposed benefits of an investment are likely to be speculative and probably marginal. In fact the UK Department of Trade and Industry have stated that “the main barriers (to the further use of information technology) are the lack of appropriate cost-benefit techniques.”
Interestingly enough, when discussing the purchase of a new IPS device a couple of days ago, TCO and ROI weren’t considered. The decision was made based on an estimation of the risk and the supposed affinity of the control towards mitigating that risk. I wonder if a different decision would have been made if we’d looked more closely at the value of the investment in pure financial terms or whether it’s even possible to do so.
McAfee think that it is. They have written a paper entitled Network Intrusion Prevention Systems Justification and ROI where they take a single case study and claim that for “any organization, the cost of (managing a malicious attack or virus outbreak) will far outweigh the cost of purchasing, implementing, and managing the IPS.” The paper goes on to claim that for a $200k investment made in IPS, the company in question makes an annual saving of nearly $4million. I think such data is misleading at best, absolutely wrong at worst. There is no accounting of risks that are most likely to have a significant impact on the business in question, only some tenious reference to the “average cost of the Slammer virus.” In claiming annual savings, McAfee are also not taking into account the future, and very dynamic, risk environment.
Of much more value is a study made by Charles Iheagwara back in 2004, in a paper entitled “The effect of intrusion detection management methods on the return on investment” (available in the May 2004 edition of Computers & Security Journal). Iheagwara states
To effectively analyze and calculate the IDS ROI, there is the need to have a sound understanding of the environment where the IDS is deployed including, at a minimum, the business practice, and network architecture and asset values.
The paper also takes into account intangibles such as goodwill and opportunity costs, “Although intangible factors inherently introduce subjectivity into risk and return analysis, it is nonetheless an important step to consider intangibles before one can arrive at a more meaningful calculation of the ROI.” The conclusion here is that
a positive IDS ROI is attainable with an effective deployment technique and optimal management approach.
The question becomes one of whether or not any of this is going to figure in the decision whether or not to purchase the device. In my opinion financial analysis must be performed for information security related projects and that this analysis should be subjected to external review from non-IT personnel. Projects will gain a lot of credbility from being described in terms that demonstrate their value to the business. My own conclusion from a paper I wrote some time ago were that the best way to make a financial judgement in support of a particular security control is to use the following formula (from S.A.Purser S,A, “Improving the ROI of the security management process”) :
Taking risk into account provides more relevant assessment of a security related project. It is this methodology that I would recommend to the business when making an assessment of the value of security projects