Case study: European Medicines Agency and CAST

The European Medicines Agency, which is responsible for evaluating medical products across Europe, expects to cut millions of Euros from its software development costs after rolling out software to automatically identify programming errors in computer code.

The European Medicines Agency, which is responsible for evaluating medical products across Europe, expects to cut millions of Euros from its software development costs after rolling out software to automatically identify programming errors in computer code.

Early trials show that the Agency, which spends 20 million euros a year on IT and supports 21,000 users across Europe, has been able to bring in more projects on time and to budget since it deployed the software from CAST Research Labs.

The EMA expects the project to pay for itself in less than a year by reducing project over-runs which can cost up to 200,000 euros a month for large projects.

Hans-Georg Wagner (pictured), CIO at the EMA, said CAST would help the agency's programmer's identify code errors before software went live, significantly cutting the cost of remedial work.

"I made a fairly strong statement to the CEO and the CFO that I was confident that once in regular use, we would have a return on investment within a year, given our total development spend," he said.

Click here to download CAST Research labs' global benchmarking study. (requires registration)

The EMA is responsible for running 110 information systems, which cover medical product regulations, register clinical trials, inspections on drug manufacturers and clinical trails.

It has nearly 70 projects under development, ranging from creating new information systems to making major enhancements to its existing information systems.

"About two years ago it became clear that in spite of following all the best methodologies, including Prince II, we started delivering software that had significant quality issues. We know that any quality issue costs seven times as much to fix in production code than at the design stage. I decided as CIO that we needed to do something about software quality," he said.

At around the same time, Wagner said he received an approach from CAST UK, with a proposal to test CAST's analytical software.

"I remembered having tried out cast in my previous job working for the European Commission in Luxembourg. I decided to run a proof of concept with one of our most difficult systems," he said.

The EMA tested the programme on its EudarVigilance database, a European-wide system to record adverse reactions to drugs. The system uses a variety of different programming languages, including Visual Basic, SQL and PL SQL.

"We drilled down into detailed issues and confirmed what we knew from anecdotal experience. And we found that even in areas we had not looked at, it was providing us with useful information on which areas were weak, such as documentation," he said.

The real benefits of CAST, however will come when EMA rolls out the software to systematically analyse the source code of projects as they are under development.

"What matters to me more than the absolute measure of software quality is the trend. As CIO I would like to see the trend in the number of errors going down rather than up," he said.

EMA is using CAST to actively identify poor programming practices in 12 applications, and on another 6 applications in a more reactive way.

Although it is early days, Wagner said he can see a clear difference between the projects that make use of CAST and those that don't.

"What I can say is that in systems where we have not used CAST, there can be problems. In one case we were one year late rolling out a new version which was eagerly awaited by the users. The cost of that one case along is one million euros. That alone exceeds the cost of CAST," he said.

Ultimately the project will also help the EMA to ensure that critical medical databases do not suffer from unnecessary downtime - a problem that in extreme cases could be a matter of life or death.

"Some of the functioning at the extreme end is essential to protect life. The earlier you can investigate adverse alerts, the better. You don't want to be surprised by learning there is a problem with public health in the press," he said.

"If the information systems are not available, people can't run signal analysis and trend analysis. Its essential that the data is good quality and the analysis is correct. So there are a lot of constraints on software quality."

The biggest challenges in rolling out CAST is not technical, but in persuading developers that CAST will help them develop better code, rather than show-up their failings, Wagner revealed.

"There is a certain amount of resistance from development teams and project managers who feel this is a tool to measuring their performance. I had to do quite some work dispelling this fear. It was the usual change management challenge," he said.

EMA plans to expand its use of CAST in 2011. One plan is to use the package to document legacy code, where the documentation is poor or non-existent, to provide better estimates of the cost of change requests.

"We frequently get requests from users following changes in legislation to change our data model. The CEO wants to know how much this is going to cost us. We know all the standard methods for estimating costs, but I would feel much more comfortable using CAST to find out how many lines of code would be affected."

IT directors from other European agencies, who meet twice a year to exchange ideas, are showing interested in using CAST, said Wagner. At least one agency has begun trials.

"We believe that CAST does what is printed on the box and does it well. We are very happy with the software and very happy with the support," he said.

Related articles:

$1m worth of bugs in average app, study finds

The software that costs the most to fix once live, revealed

Why do IT suppliers write better software for themselves than their government customers?

Read more on Software development tools