As an advisor it would be a simple task to sit and criticise other people’s work without offering constructive advice, but the “restaurant critic” approach has never been one I am comfortable with – not even in print.
It is often assumed that advisors have little experience of operational security, since they tend to be involved in setting up complex programmes, but as a partner at KuppingerCole UK we have adopted an ethic of “practise what you preach” and “talk about what you know (and don’t if you don’t!)”. My writing on governance, risk management, data security and operational security is entirely from personal experience, not to mention the many mistakes made along the way.
Anyone involved in large security projects or programmes will be familiar with IT security health checks (ITSHCs) – the vulnerability assessments and configuration checks required for assurance of IT infrastructure before go-live and continued operations, usually on an annual basis.
ITSHCs tend to take two different flavours: networks and software. Within these areas, networks can be tested for vulnerabilities and patches, or penetration tested. Software can be reviewed at the code level or checked for how easy it is to exploit. It is recommended that all of these methods of testing are used, but proportionally to risk and requirements. After-the-fact software code reviews are costly and unnecessary if a proper secure development lifecycle (SDL) is followed showing proper software security processes, versioning and peer reviews along the way. Test time can be optimised with modular design, based on well-tested components. Model-driven software development keeps code consistent when changes are made.
More on IT projects
Imagine the scene: it is the closing stages of a project, a system needs testing. Go-live is in four weeks, infrastructure is set up, other system and user-acceptance testing is also taking place. An ITSHC is required to make a risk judgment and sign off a project.
Scoping a test takes a day and requires an architect, a security consultant and a tester to be available. The scope then needs to be agreed with the project. Arranging a meeting for this usually takes a week, plus a day to discuss it with the relevant technical and management resources. Once the scope is agreed, changes need raising, including booking time in environments, getting access to environments and datacentre.
I was once advising on a medium-sized project covering two datacentres, two client sites, multiple network devices and a disaster recovery (DR) setup which was split into five separate phases. This is not unusual or large. The changes took four weeks to agree, raise and get into the change control system, mainly due to clashes with other project work. This took six to eight days of a consultant’s time to arrange.
The change board needs to review these change requests. Typically change lead times are measured in weeks, not days. We are already looking at three to four weeks before the change board takes place. In my experience lead times for environment access can be as long.
A mid-sized test takes place over a period of one to two weeks, costing anywhere from £15,000 and £25,000. Vulnerabilities are highlighted in a report, listed by IP address or device name, often with little or no reference back to original architecture or security context.
Engage with a trusted ITSHC company as soon as the architecture is documented
These reports need to be seen by lines of a service responsible for vulnerabilities, and fix schedules arranged around them. The larger the project or programme, the harder this is. Who pulls together the lines of service? Who prioritises their work? Who says your project deserves the attention? As an advisor I can push these issues to the right party, typically a consultant will not, assuming it to be a project manager’s job, and is usually too busy to pick it up himself.
Assuming this phase can be done without clashing with another project stream, it can be reasonably assumed that raising a fix schedule can be ready within a week, and assigned to the correct lines of service with timescales.
This all needs to be managed and reported back to management, with progress shown over time, and fix schedules agreed. This requires preparation, a meeting and a day of a consultant’s time. It should not be managed by the consultant though – this is the job of a specialist project manager, or testing manager.
More on security
If each of these resources costs approximately £1,000 a day, the project we are now standing at costs of £30,000 to £40,000 and 10 weeks of elapsed time from initial scoping. This is all before a single fix has been put in place.
And we wonder why people complain about IT Security costing money and slowing projects down…
I stated at the beginning that this was four weeks to go live, and yet on these timescales we are conservatively ten weeks down. This is not atypical of testing timescales. Starting testing three months before go-live would be impossible in most cases, as the infrastructure would not be ready.
So, where did we go wrong? My advice to anyone in the same position is to state the truth, as it will save time in the long-term. If the project team have had any testing experience themselves, they will appreciate your knowledge and honesty.
- When starting an enterprise level project, state in the initial proposal that the testing phase will take three months, that it will be staggered around the go-live date, and not everything will be completely fixed before go-live.
- Engage with a trusted ITSHC company as soon as the architecture is documented, let them review it and comment. When you come to the test, the scope should be ready and reviewed.
- Have configurations ready to come off devices and tested while changes are set up.
- Ensure you know your change managers, let them know two weeks in advance that testing is due.
- Likewise the environment managers, book in a month in advance if you can, three if possible, the sooner the better with these guys, their life is geared to planning.
- Stay close to your project and programme management team. Let them know the plan, tell them the risks, the changes, the hold ups – they may be able to help in ways you aren’t expecting. Your change board and lines of service are heavily influenced by them, use that to your advantage.
Robert Newby is an analyst and managing partner at KupingerCole UK
This was first published in September 2013