Managing IT projects: when things go wrong

No matter how thoroughly you plan, unexpected events can disrupt IT projects. Find out some expert survival tips

Imagine this: you are managing IT service for a customer, which includes regular update of records in real time. Your datacentre has a backup centre and you can switch the service within microseconds in case of a problem. Updates are backed up as they happen.

On Friday you had a power outage and service switched to the backup centre with no problems. On Monday, the customer informs you that some transactions from the weekend have mysteriously disappeared.

Investigation reveals that configuration of the service at the backup centre, which follows a weekly cycle, was not complete before the outage occurred and, as a result, not all transactions were backed up properly. It takes the work of 20 people working full-time for five days to sort it out.

This scenario is based on events that did occur, and it was a combination of bad luck and “bad luck”. 

And while the sun and moon endure, luck’s a chance, but trouble’s sure

Houseman, Last Poems

That’s life; such things happen all the time – no matter how thoroughly you plan, events – black swans, blow-ups, whatever you call them – will occur, and the project manager and project owner have to deal with them. Harold Macmillan, British Prime Minister in the 1950s, when asked by a journalist what he most feared replied, "Events, dear boy, events".

We should have thought of that

Given that the majority of experienced project managers take planning seriously, they account for expected events in their project plan; events they think might happen are accounted for when managing risks.

Trouble, however, is the other stuff, the "damn, we should have thought of that" or "we never believed that would happen". Let’s look at each one.

If your reaction is "damn, we should have thought of that", you already realise that the event could have been planned for or managed as a risk. You can only eliminate these by preparing more thoroughly than you ever thought you should.

How to anticipate risks

Look for aspects of the project nobody in the team is familiar with. Seek out these aspects – legal, technical, political and business-related – and find people in your organisation with some experience in that area.

Learn more about troubleshooting

Download a free extract from Jeff Morgan’s book, Managing IT Projects for Business Change. Includes a discount code for Computer Weekly readers.

Ask them about what can go wrong – and take their advice into account, either in the plan (where it is most likely to happen) or as a risk (if you believe this less likely). If you can’t find someone with experience, mark this aspect as a risk with unknown characteristics and be on your guard for it.

If your reaction is "we never believed that would happen", you are outside the normal run of events. It might be something completely new or, if previously known, be on a greater scale than expected.

The shock of the unexpected

As an example, one case I recall was where the server power demand was so great that portable power generators had to be rented and then (as the servers were sited on an industrial estate) other estate occupants had to be compensated for the increased noise. 

The need for more power had been anticipated, but no-one guessed the scale of demand because no-one realised the projected capacity requirements were way short of actuality.

With such events, after the initial shock, your job is to clean up as quickly and as well as you can, and then decide on the longer-term action. The range of consequences is wide, from a relatively small change in scope – when everyone breathes a sigh of relief – to an existential threat to the project and perhaps even to the organisation carrying out the work.

You may need a good lawyer

If the consequences are only to project scope then it may be that a small amount of pain (taking a financial hit, for example) will do. In the worst cases, senior managers may need to get involved because you have some major retrenchment to do. You may need to get a good lawyer.

You may (as I have in the past) need to trawl through years of documents and emails to establish lines of defence in case negotiation fails. You – or more likely your senior managers – may decide that to continue the work is to throw good money after bad and it is better to take the consequences of quitting – even if this involves reputational damage.

There are two messages to take on:

Be prepared – use all the skills and experience you can muster, within and outside the project team, to plan for trouble.

Learn over the long term. Don’t look at each project in isolation – the experience of one project will help others, current and future. As trouble strikes, you should learn how to cope and anticipate. The organisation will benefit for it and will in the long term become more resilient; it will also save money – and maybe even its own existence.

Trouble is the opportunity to learn

As an organisation, don’t forget what happened and what the consequences were. Aim for the long term; replace reflecting "we never believed that would happen" with "we considered it and were prepared for it."

Jeff Morgan’s 30+ years of management experience include projects such as the London Stock Exchange ‘Big Bang’ and the UK National Health Service Programme. He guided projects that were "off the rails", including some in serious dispute and only one step from the lawyers. Jeff had significant responsibility for CSC’s project management method as an author and subject matter expert, and for several years, led its design. He also set out the principles for CSC project manager development and helped develop courses for experienced project managers.


Read more on IT project management