David Lacey's IT Security Blog

I’ve tracked with some interest the developing media coverage triggered by the British Airways uniform controversy. I always recommend that anyone working in security takes a close interest in any external organisation that appears to be going through a crisis. And by “crisis” I mean an incident that begins to spiral out of control, seriously threatening the organisation’s revenue stream or ability to operate. There is no better learning vehicle than to observe and learn from the actions of others. That’s because it’s much harder to see things objectively when they happen within your own organisation.

The BA case demonstrates a classic trap in crisis management, i.e. tacking the apparent trigger of the crisis rather than identifying and addressing the actual cause. A classic precedent was the Brent Spar disposal. In that case, the root cause of the crisis was Shell’s communications style, rather than the soundness of their environmental disposal case. So the more they argued, the worse the crisis became. Shell eventually had to give in to the media pressure and start to transform their communications, with more focus on listening to what customers actually thought about their operations.

In the BA case the root cause of the crisis is clearly the negative public opinion about their uniform rules. You simply cannot argue against such perception. It will always win the day. Of course all of this seems obvious after the event. It’s much harder to think strategically when you’re in the thick of a crisis. Unless of course you’ve taken the trouble to study the lessons of previous crises, which, unfortunately, not many people bother to do.

External networking is one of the most important components of an effective security function. That's because security managers need to draw on outside knowledge and skills to identify new threats, interpret new compliance requirements and import best industry practices. Access to a trusted security circle is even more important in a crisis or disaster, when external intelligence can be crucial, and you might well need a little help from your friends. That's why the need for regular contact with authorities and special interest groups has always formed part of the ISO security standard. Today it's even more important, as security managers grapple with increasing compliance requirements, faster patching cycles and proliferating product choices. These problems are particularly relevant to companies in the Retail Sector who need to meet the demanding requirements of the Payment Card Industry (PCI) Security Standard.
Those who experienced the aftermath of 9/11 in New York will appreciate the benefits of such networks. The US ISACs (Information Sharing and Analysis Centres) provided a valuable communications channel to enable affected organisations to share intelligence, ideas and facilities.

For all these reasons I'm a keen supporter of the valuable role being played by WARPs (Warning, Alert and Reporting Points). These networks refresh the parts of Government and Industry that central security authorities and established security circles cannot easily reach. And that's essential because most of the security risks to our national infrastructure are not confined to a small circle of critical agencies. They're scattered across the multitude of small public and private sector organizations. And as time goes by, the effectiveness of our security outreach will increasingly define the risk posture of UK plc.

WARPs were set up by NISCC (now CPNI). There are now 18 of them across various sectors and they serve as a unique, trusted network for exchanging intelligence, sharing best practices and coordinating incident responses across the UK. The WARP Programme is currently being reviewed to ensure its future effectiveness and sustainability. One thing is certain. We need a lot more of them.

Of all the security clubs and associations, the one that impresses me most is FIRST, the Forum for Incident Response and Security Teams. Why? Because it's focused, born out of real business requirements and it's highly selective, i.e. you have to be sponsored and audited to gain membership. FIRST is not a club that exists to make an income for its organizers. It's an international community that serves a real purpose: helping Government, Industry and Academia to respond quickly and effectively to new security threats. So I have no hesitation in recommending that you book a space in your busy diary to attend their Annual Conference in Seville in June. Even if you can't make it, I'd recommend you check out the excellent Global Security News feed on their Web site, which must rank as one most comprehensive sources of up-to-date security information.

Yesterday Tiscali, a leading UK ISP, admitted that they were "experiencing issues with outbound emails" having been targeted by spammers which resulted in other ISPs blocking emails sent via their service. For the past few days they have been installing new hardware to address this problem, which seems to have caused further disruption to customer services.

Of coures it's good that Tiscali are admitting to - and responding to - an identified security issue. But reports of problems with Tiscali outbound email have been circulating on the Internet for more than a week. And the nature of the problem seems to have been such that customers did not been receive error messages informing them that their outbound emails were disappearing into the ether. So many important business and personal communications may have been lost for good.

There are three main learning points from this incident. The first is to ensure that you have fallback arrangements for your email services. The second is to ensure you have controls to identify when emails are not being received by recipients. And the third one is for service providers to realise that customers should be informed immediately of service problems so that they can invoke contingency plans.

Yesterday I took part in an excellent Symantec event in London on Data Leakage. This is a very hot topic and one that seems to be getting scarier by the day, with regular media reports of incidents with high financial and reputation impact. Things have certainly changed from the heady dotcom days when availability was all the rage, and confidentiality was seen by many as a relic from the past. Encryption of data at rest was rarely encountered (though we did it at Royal Mail to protect our customer's credit card details). But regulatory compliance and high-profile incidents have since transformed the security landscape and confidentially is back with a vengeance and rising to the top of the security agenda.

I've been pointing out for some time that the costs of security incidents are both understated and rising. Just look at some of the recent costs associated with data breaches. Nationwide was hit by a fine of almost £1million, arising from the loss of a single laptop. And that's on top of all the operational costs and reputation damage. TJ Maxx have already reported a fourth-quarter charge of $5 million to cover the costs of investigation and remedial work from an incident in which details of more than 45 million credit cards were captured by hackers exploiting security weaknesses in their infrastructure. Many pundits expect the cost to go much, much higher. Most of this speculation arises from recent research by analysts and institutes, such as the Ponemon Institute, which indicates that the cost of data breaches is now well in excess of $100 per compromised record, suggesting that the overall consequential costs of the TJ Maxx incident might run into billion of dollars.

Of course all of this is speculative because we can't know, measure or separate out all the current and future costs of an incident. But there are a lot of direct and consequential costs arising from data breaches, including for example the costs of investigations, remedial work, lost customers, loss of brand value, additional regulatory demands, fines, lawsuits, PR costs, and the costs of re-issuing credit cards. Not to mention the overall impact on e-Business from customers switching to cash payments. But one thing is clear. The risks and impact will continue to rise until organisations achieve much higher levels of security, including tighter platform and network security, better staff awareness and more aggressive auditing and monitoring of operational processes.

I enjoyed watching the latest Die Hard movie. It’s excellent entertainment. And it's all about cyber terrorism. So it’s a must for anyone working in the Critical National Infrastructure field. Of course, it’s impossible for anyone working in security to watch a film like this without comparing every small detail to real-life experience.

I was particularly interested in the design of the FBI’s cyber security crisis facility. Just like 24’s CTU, it has lots of colourful screens in darkened rooms but seemed a little short on decent facilities for top-team discussion and decision-making. I assume the set was probably modeled on the National Counter Terrorism Centre in McClean Virginia. This is an impressive facility with an interesting lay-out of well-spaced desks in a large room with a central focus. A very effective layout for managing fast-moving operational issues. And very different from the recently made-over White House Situation Room, which has a layout designed to facilitate centralised discussion and team-work.

It's not clear how much benefit colour displays offer over traditional flip charts and white boards. But you can’t beat a good “wow” factor. And there are some breathtaking environments out there that are actually used for crisis training. The most impressive one is the Louisiana Immersive Technologies Enterprise (LITE) complex at the University of Lafayette, which boasts three-dimensional immersive visualization cubes with high definition walls, floor and ceiling, supported by high-speed networks of high performance computers. Just imagine what you can do with all that.

I'm a keen student in the art of designing environments team structures for crisis management and emergency response. These two functions are often confused. But they are quite different. The former is focused on preserving the longer-term intellectual assets of the organisation. The latter is primarily concerned with short-term physical events. They require different environments and team compositions. And small factors such as the size and shape of a room do have a major impact on team dynamics. It’s a shame that so little attention is paid to room design. Because the real art of crisis management is maximising the value of every asset at your fingertips, whether people, information or systems. And you need the right surroundings to achieve this.

You can’t open a UK newspaper or switch on the TV news without getting extensive coverage of the devastating floods. That makes it a very good time to review Business Continuity Plans.

There’s always a spike in business awareness and interest in Business Continuity Planning following a major incident. Especially one that attracts extended media coverage. I first noticed this following the IRA bombs in the early 90s, which led to a step change in priority and budget for BCP in London based organisations. I also recall the floods in Holland in the mid 90s generating a surge in demand for BCP consultancy thoughout the following year. Similarly, the terrorist outrages in New York and London also prompted many organisations to re-think their contingency arrangements. And there was also a large increase in emergency response and crisis management activity across Texas following Hurricane Katrina.

So, even if you’ve not been affected, it’s worth visiting business units to review current BCP arrangements. Because they're likely to be considering just what they would do if confronted by such circumstances. But in six months these thoughts may have been superseded by more immediate business priorities.

With many parts of the UK still under water and further bad weather forecast, it’s remarkable timing that a new disaster film, due out shortly, is based on London being flooded by a storm surge. The film is based on a book by Richard Doyle. I haven’t read the book it but the Web site is worth a look, as it contains some useful background on the risks of such an event. Amongst other things it points out that the Thames Barrier was built to a risk level of 1 in 1,000 years. The odds against a flood might be long, but it can still happen. We also have the prospect of global warming loading the dice. And it’s not impossible for the Thames Barrier to be put out of action, by a terrorist incident for example.

The business impact from a London flood would be massive. So how well are organisations prepared? In my experience, not very well. Several years ago I took part in a crisis exercise simulating such an event. It was difficult to say the least. Without advance planning and immediate access to a diverse range of information, it’s impossible to quickly identify and assess the impact on a range of offices, outlets and staff operating inside a specific geographical area. It’s also extremely difficult to evacuate and secure premises, vehicles and assets within a short period of time. Evacuated premises could be inaccessible for weeks, so critical or valuable assets need to be duplicated, relocated or secured. And that’s not easy if staff are unavailable, roads are blocked and public transport is suspended.

For any organisation with offices or business outlets operating near the Thames, I’d strongly advise developing a specific business continuity plan. A London flood is not likely but it’s possible. And you need a lot of advance preparation. You won't be able to wing it on the day.

With Business Continuity at the forefront of my mind, I was interested to read that Bruce Schneier's recent comments on pandemic planning had attracted some criticism from journalists and analysts. Surprisingly, he suggests that it "really isn't the sort of disaster worth planning for", i.e. because the scope of the disaster would be so large, that business continunity planning would be ineffective. He believes that “the proper place for bird flu planning is at the government level...real disasters don’t exactly match our plans, and we are best served by a bunch of generic disaster plans and a smart flexible organization that can deal with anything”. It’s an interesting view and he’s right to some extent, in that it’s often better to focus on good response skills, rather than prescriptive plans. But not every crisis is the same. And many, like an Avian Flu pandemic or a London flood, require considerable advance planning.

Planning for a pandemic requires a detailed analysis of the vulnerability of critical business processes, supply chains and essential services. Organisations need to single out key staff required to support critical business processes and take appropriate steps to enable them to carry on working as long as possible through the crisis. They should assess the vulnerability of suppliers and outsourced services, and identify whether changes are required in business strategy or operations. They should consider the impact on IT services, especially communications, and assess whether they should extend or upgrade home working capabilities. They should consider office layouts that reduce the spreading of germs. They should consider the impact on the future business environment and adjust their investments accordingly. And of course they will need to prepare the HR function for dealing with large numbers of absent personnel, some of whom might not return. Pandemic planning is a big job, and it shouldn’t be dismissed lightly.

Schneier and others are also wrong in suggesting that business organisations shouldn’t bother planning for huge catastrophes, such as nuclear wars. In my view, you can and you should. The Shell Group, for example, managed to operate through many conventional wars, such as Biafra. They also prepared themselves for a Cold War conflict by maintaining secure, remote, archive sites in separate hemispheres. Good crisis planning is about addressing the unthinkable. That’s how you ensure the long-term survival of your organisation.

My recent posting on pandemic planning prompted a comment suggesting a Top 10 list of actions. I couldn’t resist the challenge. Each organisation is different and requires its own specific action plan. But there are many common actions and principles. I’ve pitched this at organisations rather than citizens because the latter is a job for government agencies. And I believe they have most of this in hand.

My posting earlier this week on the costs of incidents created a few stirs, the most interesting one being an email from the excellent Ponemon Institute, who have been the source of many highly-publicised claims about the costs of data breaches.

Estimating the potential cost of security incidents is fundamental to corporate risk assessments and the resultant business cases for security spending. It’s clearly vital that security professionals have a sound model for estimating potential business damage. And the Ponemon Institute research is the most authoritative basis for this. Because it’s based on up-to-date analysis of real incidents. The Ponemon research also provides useful metrics for business cases, such as the total recovery cost per compromised customer account. Every security professional should become familiar with this research because it’s central to the justification for the resources and budgets needed to mitigate the risks of data compromise.

The difficulty of course is translating past research findings into future reality, especially when the scale is different. Such as in the recent incident at TJ Maxx, where many of us were tempted to extrapolate figures based on thousands of compromised accounts into estimates based on millions. And how well did we do that? Not at all well I'm afraid to say. Most analysts simply multiplied the historical average damage per account by the number of compromised customers. This projected a hit of several billion dollars prompting a wave of doom-laden warnings.

We should have listened to Larry Ponemon. Because he actually published statements at the time pegging the projected cost in a range of “hundreds of millions” of dollars. The TJX Group initially claimed a total cost estimate of around $25 million, but recent updates have inflated that figure by ten times, square within the range projected by Ponemon. What the pundits overlooked was the fact that TJ Maxx was an exceptional case. The breaches studied by Ponemon were in the range of a few thousand to a quarter of a million. The TJ Maxx incident involved more than 45 million cases. But, as Ponemon point out, many of the costs associated with data breaches are not fixed ones. The larger the breach, the smaller the resulting per-record number.

So well done Larry for getting the projection right. And there’s a clear lesson for us analysts and pundits to be a little bit more cautious in translating research into reality.

I was interested to read Tom Ilube’s comments on the BBC Website about the failure of Northern Rock’s systems to cope with the recent exceptional demand. As one of the pioneers of online banking (he was Egg’s CIO) he should have a good perspective of what is achievable in designing systems that can respond to unanticipated peaks in demands. As Tom points out, building systems that can cope suddenly with a completely unexpected burst of perhaps 10 or 20 times their normal processing volume is notoriously difficult to do, but by no means impossible.

So what went wrong? Was it financial constraints, a lack of planning or perhaps a deliberate ploy to slow a potential run on the bank? In such cases I generally assume a cock-up rather than a conspiracy. Organisations are not good at planning for extreme circumstances. It’s outside their experience. But today’s business world is much more volatile and fast-moving, and the pace will continue to increase, because the faster the business cycle, the higher the revenues.

As Tom suggests, every bank chief executive should ask their IT director today "If we are hit by 10 times our normal customer volume tomorrow, what will happen to our online banking system?" And based on my experience of business continuity planning, I’d add a further demand. “Prove it”.

A new survey by Symantec suggests that more than nine out of ten UK organisations carry out full evaluations of their disaster recovery plans but almost half of the tests fail. Should we be surprised by these figures? Absolutely not. In fact the figures are quite encouraging. We must be making progress. Because I've encountered an awful lot of critical business processes without proper continuity plans. And where they do exist, they’re often incomplete, out of date and generally fail when tested.

Business continuity planning is a thankless, time-consuming and messy activity. It’s not an exact science. More of a painfully slow journey in progressively improving the disaster response process, with frequent setbacks when any restructuring or reengineering takes place. You can’t outsource the work to consultants because business managers have to manage the process, so they need to be fully engaged in all aspects of the work. That's assuming of course that they can be persuaded to set aside the time and budget to stay on top of developments.

And it’s not uncommon to discover that there simply is no viable fallback option, either because of limitations in infrastructure design or the sheer expense of a replacement facility. Of course it’s relatively easy to replace modern IT infrastructure, and that was probably the context of this survey. The hard bit is buildings, plants and people.

The art of crisis management is to think forwards and aim to stay ahead of the media, anticipating negative coverage and taking steps to mitigate reputation damage. That’s why it pays to be honest about facts that might emerge at some future stage, and to have prepared responses for any anticipated speculation or spin by other stakeholders.

So I was a surprised to read in the Boston Globe that a group of banks are claiming that 94 million accounts (more than twice the original estimate) were affected in the theft of personal data from TJ Maxx. That’s a staggering number, suggesting a much higher potential loss than previously estimated. It will no doubt generate a new wave of damaging publicity, despite the fact that the company seemed to have successfully drawn a line under the incident without suffering any serious impact on sales.

The key learning point is that, when in a crisis, avoid leaving room for future, sensational claims or speculation that might undermine your hard work in rebuilding your reputation. In particular, pay attention to the second rule of holes: if you’re in one, stop digging.

Alan Stockey, a fellow security professional, kindly sent me a copy of his 2007 Christmas poem. It follows on from previous classics such as "Twas the night before audit”, "Regulator's coming to town" and "Oh what fun it is to decommission and not pay". This year’s features the Grinch, a character who, with a heart two sizes too small, might not be completely out of place in a contemporary crisis team…

The Grinch who stole crisis!

He HADN'T stopped Crisis from coming!
IT CAME!
Somehow or other, it came just the same!
And the Grinch, with his visage, eyes wide, fire-aglow,
Stood puzzling and puzzling: 'How could it be so?”
It came without testing! It came with a bang!
'It came without call-trees, pandemic or plans!'
And he puzzled three hours, `till his puzzler was sore.
Then the Grinch thought of something he hadn't before!
'Maybe Crisis,' he thought, makes our heroes plan more.
'Maybe Crisis...perhaps...should be welcomed - encore!'
And what happened then...?
Well...in Cities they say
That the Grinch's small plan
Grew three sizes that day!
And that minute his scorecard turned Green – what a score!
He attended all micro-drills, failovers and more
And he brought back the desktops & Home-working with glee!
And he...
...HE HIMSELF...!
The Grinch led C.M.T!

AS 2007

A colleague recently pointed me The Breach Blog, a useful site that collects reports of data breaches. It’s fascinating to browse through the category archives and see just how many household names have been named and shamed. In today’s networked world there’s no hiding place for data breaches.

As Abraham Lincoln once put it, there are times when "the necessity of being ready increases". Given the relatively high threat of a terrorist incident disrupting vital energy supplies, it’s prudent to be ready to respond quickly and effectively. So was it madness or good management to evacuate the Safe Scandinavia, an offshore accommodation barge, last weekend?

According to press reports, the evacuation was triggered by rumours following an overheard conversation in which a woman recalled a dream about a bomb on board. The exercise is estimated to have cost in excess of a million dollars. “Madness” and “ludicrous” suggested some observers. I disagree.

If there’s any doubt then you have to play safe. And the evacuation would have been a good exercise. Regular call-outs and drills are an essential element of emergency response planning. And much of the cost will be existing employees’ time rather than additional expenditure. By all accounts the evacuation was executed successfully. That’s good emergency preparedness.

Most IT security professionals are aware of the damaging influence a major incident can have on brand value and company reputation. But it doesn't have to be that way. At this week's MISTI CISO summit in Orlando, several presenters expressed amazement that the stock of price of TJ Maxx had increased after last year's highly publicised data breach. But it was no surprise to me.

Anyone who has studied the aftermath of company crises will appreciate that heavy media coverage is also free advertising. If you handle a crisis well you will come out on top. Academic research by Deborah Pretty at Oxford University several years ago confirmed this counter-intuitive phenomenon.

Any organisation that spends millions of dollars on security consultancy and fixes following an incident gives me confidence. I'd rather trust a product from a company that's been hit and learned a lesson, rather than one that might have just been lucky. And clearly the stock markets also think that way.

Yesterday I was speaking at Reed’ Exhibition's excellent Business Continuity Conference at London Excel. The subject of my talk was “Why crisis exercises don’t work”. There was a touch of deliberate controversy in this choice of topic. I was aiming to challenge the accepted wisdom. But there’s a lot of truth in this perspective.

I should point out however that when I say “crisis”, I don’t mean local incidents, emergencies or even major IT failures. I’m referring to events that spiral out of control, overwhelming management and threatening the very survival of the organisations. This type of crisis management requires strategic thinking, logical analysis and imaginative solutions. It demands a broad, objective perspective that’s different from the one inherited from your day job. It also requires extraordinarily good teamwork. That’s already hard enough to achieve in a perfect crisis room. But it’s even more difficult to achieve across an international conference call, spanning unknown people from multiple organisations. Yet that’s the reality presented by today’s modern business environment of deep outsourcing, complex supply chains and virtual team working.

Exercises are essentially opportunities to come to terms with our limitations. Crisis teams rarely possess the full knowledge, skills and objectivity to develop the optimum strategies and solutions. Careful planning, preparation and practice do help of course. They enable us to deliver a higher standard of cock-up. And that's worth a large chunk of market capitalisation. It could be the difference between sinking and swimming following a real business crisis.

Back in 1999 I suggested that the Electronic Pearl Harbour would not happen until 2006. In fact it might have happened in 2007. The unprecedented cyber attack on Estonia last year certainly appears to have served as a wake-up call for the West. As reported in Computer Weekly, Nato chiefs have endorsed a new cyberdefence policy.

It’s the start of a long journey towards a new era of warfare. Where will it end? In mutually assured electronic outages, robot wars or David and Goliath style gaming contests? The possibilities are both fascinating and frightening.

The keynote addresses at RSA conferences are mainly vendor marketing pitches. But it’s worth taking a listen to Homeland Security Secretary Michael Chertoff’s talk at this week’s RSA conference in San Francisco.

It’s an eye-opener. The US Government has finally woken up to the realities of cyber attacks. It's building a heavyweight cyber security center, pumping hundreds of millions of dollars into cyber defences and aiming to kick federal security defences into shape.

Chertoff sees the new programme as a Manhattan project for cyberspace. It’s an ambitious initiative. He aims to speed up incident response cycle times so that attacks can be detected before they’re launched. He wants to tackle weaknesses in the supply chain such as back-doors planted in hardware or software. And he’s aiming to attract the brightest and best from the private sector to work on the programme.

It’s no less than a revolution in thinking, at least for government security. Let’s hope that something useful for industry and society comes out of it all.

Google are sponsoring their participation in oCERT, a new form of computer emergency response team style service for the open source community. The service aims to “help both large infrastructures, like major distributions, and smaller projects that can't afford a full-blown security team and/or security resources”.

It’s an interesting development, though only time will tell as to how it develops and what value it adds. In practice many of these circles end up doing quite different things from what they originally expected to do.

FIRST, for example, has branched out into running corporate executive programmes, a far cry from its original techie focus. eema dropped its original title of “European Electronic Messaging Association” because it was too restrictive and now aims to address anything to do with identity and e-security. The Open Group has transformed itself many times since its predecessor, X/Open, was set up the early 1980s to agree a non-proprietary operating system standard for minicomputer vendors.

I wish them well. In the security world you live on your contacts. We can’t have enough of these circles, especially open, free ones.

This time it's the credit card details of up to 38,000 customers of clothing retailer Cotton Traders that have been stolen according to the BBC News.

The firm claim to have upgraded their security. It's tough for those that might have been affected. But it makes you wonder if you're better of with a company that's been hit and sorted itself out, or one that has yet to be caught out.

Certainly I'd be more comfortable in future dealing with Cotton Traders. It will be interesting to see how the market responds.

I'm fascinated by the subject of data integrity: the growing challenge of sorting out the truth in a world dominated by mistakes, disinformation, FUD and spin. I don't believe everything I read without compelling evidence and a strong dose of reality, though I must admit that it sometimes seems that, increasingly, reality can be stranger than fiction. In particular I'm amazed at how quickly news items can go global, and, in some cases, viral. 
 
I was surprised, for example, to read the recent, highly publicised story, sourced by the Sun, about an MI6 camera being bought by an individual over eBay. Now I have absolutely no idea if it's true. But whether or not it is, it just doesn't seem credible to me. But it's been quickly taken up and reported by quality newspapers worldwide.

I can't help having some doubts about this story. Firstly, it seems very odd that MI6 should be using cheap consumer cameras for operational work, and even odder that they should dispose of them through eBay without checking them out. They're not that stupid.

Secondly, it seems surprising that it's been so solidly attributed to MI6, though it sounds as though the material could just as easily been sourced by any other agency with an interest in the subject area. 

Thirdly, the description of the contents sounds unusual: fingerprint information, log-in details for the Secret Service's computer network, with a "Top Secret" marking. They don't sound like likely subjects of photographs. Just take a look at the images on the Sun's website. Do they look natural?
  
Yet we have a major crisis. Perhaps someone in government has informally confirmed the facts. Otherwise I would find it hard to believe that anyone would believe a story like this coming from an average man in the street.

And that's the real problem. We will all assume that someone in the media would have taken the trouble to substantiate the story. In the absence of any indication to the contrary, we will tend to believe what we read in the press.

The most interesting aspect of the recently reported data breach at Heartland Payment Systems is the relatively light press coverage. The full scale of the breach has yet to be established, but it has been suggested that it might run into tens of millions of credit and debit transaction details, making it one of the largest data breaches reported so far. 

Specialist publications such as SC Magazine covered the story, as did security bloggers such as Brian Krebs and Stuart King, but there has been surprisingly little mainstream media attention, given that such stories certainly appeal to the press. That might of course be attributed to the timing, coinciding with coverage of the new US Presidency and the worsening state of the global economy. Burying bad news is a classic tactic for minimising reputation damage.

Even better is to apologise and offer compensation to your customers. The TJX Group have certainly got this right, rewarding customers with a special sale offering a one-day "Customer Appreciation" sale in its US and Canadian outlets to express appreciation for their customers' continuing loyalty. Saying sorry with a January sale is a smart business move.   

Good management of the aftermath of breaches enables crisis-hit organisations to come out on top. That's important given that the sophistication of modern attacks continues to outstrip the capabilities of traditional countermeasures and compliance requirements. There are many more breaches yet to come. All organisations with sensitive or critical data need to ensure that they are well equipped to manage a potential crisis. It's vital to long term business survival.   

The current crisis of public confidence in UK Parliament, triggered by the publication of MPs' expenses records, demonstrates three interesting and very important lessons of crisis management. They are worth noting, as they play a part in all major crises. 

Firstly, there is the unexpected, unprecedented level of public and media rage. As Dr. Peter Sandman, a long-standing expert on risk communications once put it: "The engine of risk response is outrage". The need to manage extreme reactions should always be taken into account when planning any crisis response. 

Secondly, there is the invisible culture that surrounds the crisis organisation, blinding them from the true nature of the crisis. They cannot see anything wrong in their long-standing practices. Good crisis management requires an objective perspective. The crisis team needs to see events with the eyes of an outsider.   

Thirdly, there is the natural tendency to focus on the trigger of the crisis, rather than the underlying cause. The legality of MPs' expenses claims, and the rules governing them, are not the real issues. It's the perception of greed that outrages the public. And that demands a much more radical response.

Unfortunately when organisations are in crisis, they tend to operate on gut instinct guided by wishful thinking, instead of logic and expectation of the worst possible outcome. It's more comfortable to aim to muddle through, rather than confront difficult issues. But a crisis is a major turning point. Things will never the same again.

Several computer publications, including Computer Weekly, are reporting that T-Mobile USA is investigating an anonymous claim that a hacker broke into its databases and stole customer and company information. Some security experts have suggested that the claim appeared to be legitimate at first glance but is likely to be a hoax.

It's a sign of the times, a reminder that we are entering an information age ruled by spin, FUD and misinformation. A single whisper can undermine customer confidence in a global service provider. But at the same time a smart crisis response can take advantage of the free media coverage to boost public confidence in an organization's management of risks and incidents. It's time for enterprises to raise their game in crisis management.   

For several years I've been preaching that the key to effective security management is to harness the power of social networks to help identify risks and prevent or respond to incidents. My book Managing the Human Factor in Information Security contains a chapter on the subject.

At this week's TED Global conference in Oxford, Internet lawyer Jonathan Zittrain of Harvard University gave a good example of how crack teams of volunteers responded to an incident in 2008 when Pakistan Telecom accidentally took YouTube offline. The service was rescued by "random acts of kindness" by unsung heroes.

Technology is spectacularly vulnerable to failures but also tremendously resilient. We just need to channel community effort in the right way. Unfortunately, they don't teach that on information security management courses.

By the way, if you want to read more about the TED conference, Howard Wright's blog has an excellent, comprehensive summary of the proceedings.

A few postings ago I mentioned the growing importance of random acts of kindness by unsung heroes in rescuing or maintaining vital Internet services. Make no mistake; this is the future of security. When things get bad, we need to call on brilliant technicians to fix things. Fortunately some of the best have an altruistic streak.

Team Cymru are a good example. You might not have heard of them, and you might wonder why a top US security outfit would want to adopt a Welsh name. But I'm pleased to report that their hearts, as well as their expertise, are in the right place. In fact they are a strictly not-for-profit enterprise outfit, but with state-of-the-art skills.

The latest Team Cymru offering is a free alerting system to pinpoint open DNS resolvers in your immediate area. DNS is the DNA of the Internet, though it's based on a devolved management model which means that not all servers are as secure as you might like them to be. The Million Resolvers Project is a reporting system to alert participants when open resolvers are detected in their local address space.

If you're interested, you should contact them to get signed up. Like many information age initiatives, security is a two-way street. But what you get back is always more than what you put in.

Making a mistake once is good for your education. Making it twice means you're not learning fast enough.

On Tuesday, Twitter suffered its second denial-of-service attack in a week. Admittedly the site stood up better, being down for only 30 minutes this time. But it demonstrates the importance of immediately beefing up security following a damaging incident that might be repeated.

Customer perception drives business value. To go down once is unfortunate, to go down twice can seem careless, but to go down three times might suggest that the wheels are coming off.

The recent two hour outage of Google's Gmail, affecting the majority of its 150 million users reflects the growing risks associated with the inevitable drift towards centralised system management.

At least Google was honest enough to issue an apology explaining that the incident was caused by an engineer's miscalculation and that they were investigating ways to ensure it did not happen again. (Mind you it's not the first of these incidents.)  That's a big improvement over O2 whose service was down for many customers during most of Saturday without any explanation.

Expect more of these crashes. Information technology is spectacularly vulnerable to tiny errors and we are building massive single point failure scenarios based on cloud computing, centralised management and technology monoculture. In response, we must all raise our game in business continuity and crisis response. 

Every now and then we have to persuade our executive to think the unthinkable. But too much scaremongering can be counterproductive. You can read a few of my thoughts on the hazards of preparing for worst case scenarios on this Infosecurity Europe blog posting.