How to survive a data breach

If you like meeting new people in the company and finally getting some face time with those who have previously just been names on an organisational chart, then I highly recommend a good data breach to bring everyone together in the same room. Usually well into the evening, for at least a few weeks at a time. Nothing brings people together like a crisis – or tears them apart for that matter.

A decent-sized data breach is likely to be one of the most stressful events you're going to experience in an IT career and, for many people, it usually marks an excellent time to consider a change of employer. Disaster recovery plans usually concern themselves with maintaining business continuity and processes; they rarely consider the effect on the corporate culture as part of the fallout of a major incident. Within information security and technology, there is one phrase you will hear echoing through the halls after every data breach.

"I told you so!"

Disasters bring change – often the very same change that individuals and departments had been pushing to happen for months, if not years. Nothing pushes loyalties to the limit like having someone work for months to try and get an issue on the table, only to have someone else run with it like it was their own idea, after the horse has left the barn.

Is it any wonder that so many companies find that the most unexpected after-effect of a breach is a sudden spike in departures of their IT staff?

Here are the things you should know about what a breach looks like from ground zero, ahead of time. Stress levels will be at an all-time high and interpersonal conflicts will boil to the surface. That dry-run disaster-planning drill you'd been meaning to do for months, but never found the time for? That one minor change request your senior engineers have had sitting on the table for weeks that consistently got deferred in favour of deploying that a cool new app for the sales team? You can bet issues like these will come to the fore!

 Here are some of the things you can do to give yourself a fighting chance:

1. Don't let security be an island - IT departments (and engineers) are notorious for the ivory tower attitude; we invented the term 'luser' to describe the biggest problem with any network. Create some meetings outside the "IT Comfort Zone" every so often; the first time you meet the company's legal and PR teams shouldn't really be in the middle of a five-alarm fire.

Bring some of the people on the ground into the disaster-planning process - soliciting input from the people who maintain the systems that support your business processes every day can give much more accurate insight into what can go wrong for your business than any book full of generic examples can. These are the people that spend their day staring at the pieces of the infrastructure that are held together with duct-tape and bailing wire.

2.Give people a place to talk - Non disclosure agreements (NDAs) will be flying left and right, stress levels will be high, and the PR and legal secrecy machine will be in full force. Many employees may have had such a bad experience with the whole affair that they move on to other employers a few months later. Keeping secrets for other people is a stress factor most people did not consider when they went into security as a career choice. If your enterprise has an employee services department or provider, consider bringing one of the councillors into any NDAs, and give employees a place to vent their concerns confidentially and legally. You'll be rewarded with fewer open slots to fill in the months following a breach.

3. Let others learn from your mistakes - If you are required to disclose a breach to the public, work with PR and legal to disclose information in a way that the rest of the world can feel like they have learned something from your experiences. Adam Shostack points out in The New School of Information Security that no company that has disclosed a breach has seen its stock price permanently suffer as a result. However the fallout of intentionally vague and misleading disclosures hang over a company’s reputation for some time (18 months later and people are still theorising over what happened with the RSA breach). Giving the industry and other companies just enough information to be able to make decisions about their own security programs cannot be overrated for its benefits to your company's reputation within the security world.

4. It gets better. Famously overheard at a security conference this year - "We're only one more breach away from our next budget increase!" There's nothing like a breach to put security back on the executive team's radar. Take this as an opportunity for new ideas and approaches, not just "we're finally getting that thing we've been asking for all year". Use the opportunity to consider new directions beyond the constraints of the "old normal". Now is the time to take "misfortune is just opportunity in disguise” to heart.

5. Test for impact, not vulnerabilities - If you are spending money on third-party penetration testing, you should be expecting more in return than the output of a vulnerability scanner and some compromised systems. Expect reports that show results in terms of impact to business operations, bottom lines and branding - these are the things your executives need to be aware of. Either you look for and determine them ahead of time, or your attackers do.

6. Don't panic! - Murphy's Law will be in full effect. The information the executive team is asking for was only being recorded by that one system that was down for its maintenance window; the report you need right now will take another hour to generate; and the only person with free hands you have available hasn't been trained on how to perform the task you need done before the lawyers check in for their hourly status update. Panic generates mistakes; mistakes get in the way of work. This advice works from both ends of the chain of command however - if your executive team is expecting an hourly 15-minute status update conference call, then that's 25% less work the people on the ground are getting done. Calm heads rule the day - set expectations early on and don't go into a disaster recovery plan that principally operates on the expectations of miracles being performed.

Conrad Constantine is a research team engineer at AlienVault