Downside of vulnerabilty testing

Few will argue that vulnerability testing is not an important part of the online product lifecycle but I was caught slightly unawares by this question in a recent meeting: if we test a product, and identify vulnerabilities how do we get the resources and budget to resolve the issues within an already tight project schedule where customer focused features are a priority and margins are tight?

Fixing security is unlikely to take priority over new features unless a very strong case can be made for doing so. So, it begs the question, when is the best time to perform testing and what should you do with the results?

If the development cycle is complete and the product is due to go into production then there can be little expectation that security issues are going to be quickly resolved on the back of a vulnerability test unless they are show-stoppers (i.e. high risk of compromised regulatory data). It’s clearly a bad time to be doing this type of testing. Conversely testing further back in the lifecycle is not necessarily going to paint an accurate picture of how the product will look once it’s in production – at those stages of the process then unit testing tools are a good solution as discussed in one of my previous postings. Testing the product in it’s production environment will provide the best report from a pure black-box perspective but that still leaves the question of how best to address the report and deal with issues.

Now, the whole arguement might sound rather academic – in other words, you’re thinking: problems have been found and they need to be resolved. But consider that up against tight schedules and tight budgets. If you can have either 10 days worth of development to resolve security issues against 10 days of development to implement new revenue generating features then which one would you choose? So once again the discussion comes down to assessing, communication of and acceptance of risk. Resolution might need to wait but in the meanwhile the least that can be done is ensuring that potential bad-outcomes are known and that risks are “owned”. It’s not a problem I’ve yet had to deal with here, however it’s certainly worthy of discussion the next time you begin a round of vulnerability tests.