From brains in jars to the corporate tower

Q&A with John Bates, CTO at Progress.

John Bates is an academic from Cambridge University and was one of the brains behind Apama. He is now CTO at Progress, which acquired Apama in 2005 for £30m.

He comes from a software development environment which he describes as "brains in jars" but has transferred successfully to the corporate world.

Here he talks to Computer Weekly about his current role and complex event processing, or CEP as he lovingly refers to it.

Q. As the CTO of a supplier what does your job entail?

As CTO of Progress Software, my primary job is to make sure the company's solutions strategy directly addresses pain points that companies face and creates the highest business value and quality for our customers.

I've learned in working with customers and partners for the last few years that the biggest item on the agenda is making companies more operationally responsive: in other words, enabling them to respond in real-time to improve their customers' experience and to beat their competitors.

As part of this, business leaders want to be more involved in projects and also make their business processes more responsive to what's happening now. Similarly, IT departments want - and need - to be more business-oriented. We need to work with businesses to be an innovation-enabler and to drive companies to be more operationally responsive.

At Progress, as well as ensuring we continually have a leading strategy, my role include responsibilities to articulate Progress's strategy both inside and outside the company and to co-ordinate any M&A activity that can help accelerate our strategy.

Q. Have you had to change how you think because of the recession?

You certainly have to understand how your customers think in order to succeed. The key thing last year for customers was how we do more with less. They needed to find ways to be more productive and make their business more highly responsive throughout the recession. One way to achieve this is to gain and act on real-time insight based on information already flowing in the business. For example, by sending real-time course and speed adjustments to ships so they use the optimal amount of fuel and arrive at the right time to be unloaded, a maritime logistics business can save millions of pounds in fuel and operations cost over a year.

Customers were also looking to add revenue opportunities through upselling and increasing customer loyalty. For example, If you're an airline and a high-value customer's bag doesn't make the connecting flight that they're on, then you might send an instant text message apologising and telling them their bag will be delivered to their hotel along with a bottle of champagne. This is where responsive business processes fits in perfectly with the aim of doing more with less.

Q. What will be the growth areas for Progress in the near future?

Progress has a real growth opportunity around operational responsiveness, further assisted by our recent acquisition of BPM provider Savvion. We've got real potential here because we've aligned our products and the whole company to address burning customer problems, problems no one else is truly addressing.

Q. Can industries other than finance utilise complex event processing technology?

Absolutely. Event processing enables any business to anticipate and respond to business patterns that can indicate real-time opportunities and threats to the business.

With our Apama products, Progress has been involved in the adoption of event processing in a number of industries. The airline industry is now starting to adopt event processing technology to help create "smart airlines" that continually anticipate and respond to operational events such as flight delays, lost bags, etc, and continuously prioritise the high-value customer's experience in the face of these events.

In telecoms, event processing has solved challenging problems such as being able to dynamically address revenue leakage and one-to-one marketing on a massive scale - for example, by sending upsell opportunities to customers based on their activity, context and location. The list of other industries where event processing is being used successfully is long, and includes logistics, manufacturing, retail, energy and utilities.

Q. Do you think out-of-control algorithmic trading contributed to the size of the financial services crisis?

The financial crisis can't be blamed on algorithmic trading. It was more fundamental issues around toxic debt that caused the problem. However, the same institutions that were involved in the crisis were also bastions of this mysterious area of algorithmic and high-frequency trading. The perception of the uninformed is that they are out-of-control gamblers risking the entire economy on a daily basis!

All that said, I have been speaking for many years on the potential risks of trading without necessary safety measures in place. Traders fear that any safety measures will just slow them down, but this is not true. Technology is available that can monitor risk exposure in real-time and block any trades that are either in breach of risk thresholds or could be considered "rogue" - without adding overhead to the low-latency nature of modern trading. Similarly surveillance capabilities are now available to banks, regulators and trading venues to monitor for potential market abuse or insider trading.

My hope is that government doesn't start banning practices associated with algorithmic trading because of its lack of understanding of them. My recommendation is to improve policing of these practices inside institutions, in the trading venues and in the regulators.

Q. Apama grew out of the UK and the skills based here but do you think there is a brain drain?

Au contraire in my experience. Progress really liked Apama's innovation capabilities in Cambridge, UK - linked to one of the world's leading universities. We have expanded this capability and have a fair sized team - including 20 PhDs - based in Cambridge, UK. We call them our "brains in jars"! Cost-wise, while not comparable to outsourcing to the East, Cambridge UK compares favourably with Silicon Valley or the Boston/New York area. And in my experience, high-performers want to stay in the Cambridge area and love working in the Cambridge Lab because they get to work with other smart people. The key is to form an environment in a location and situation that retains people, is highly valuable to a company and has acceptable fully loaded costs.

The UK is a tremendous innovation centre for the world. Our education system and culture are excellent for this. The biggest risk for the UK is the increasing cost of labour compared with the East. We have to get smart about how we deal with this and ensure we are anticipating the needs of the next generation of business in our university system.

Q. How does the UK compare to the US in terms of being the right environment to turn innovation into a commercial success?

I always describe Britons as brilliant inventors but poor business people. What global software companies can you think of that have come from the UK? More often than not we have the European arm of a US software company headquartered in the UK. Often inventions - like Apama's CEP - come from the UK. While we are OK at getting a startup to a certain level, we don't have a track record of taking them to a large scale.

However, Americans are brilliant at business - it's in the culture much more. I believe the UK should be focusing on getting business leaders with skills in sales and marketing strategy back to the UK after a stint learning the ropes in the US. If I was a venture capitalist investing in a business in the UK I would be looking to bring in a UK person with US experience to head it up.

Q. Do you think the UK should have more support for developers such as boot camps?

For our customers we have found full-immersion boot camps a great way for them to get into new technologies. We have the concept of a "sheep dip" where groups of both business and IT people come to our Cambridge Lab to brainstorm over business problems and learn how Progress technologies can be applied to help. The combination of business and IT people is important because these areas are moving closer together in the business and need to co-operate closely to solve business problems. Technologies like event processing offer tools to foster this cooperation.

At a UK level, yes I think more of a support structure for developers and other parts of the business would be beneficial. We need to think more like Silicon Valley and be up on the latest techniques if the UK is going to continue to prosper.

Q. What do you think is the biggest mark the recent downturn will leave on the IT sector?

I hope it has made the UK realise the changing nature of the world economic balance and how to adapt to capitalise. The economic downturn has accelerated certain factors such as the outsourcing of jobs to India. We need to accept that this is inevitable with some roles. However, despite the higher cost of labour in the UK it is possible to have a higher productivity, higher profit and even lower cost per person in the UK by the deployment of advanced technology. One area we're seeing is the need to deploy new business processes quicker and to be able to evolve those processes quickly as the business changes. Technologies like event processing address this need.

Q. Do you think the IT sector and business IT departments will emerge from recession stronger?

Most definitely. A wise person once told me "the hardest times yield the finest wines". Although many vines have been pruned back in this recession, it's often a necessary step for new growth. CEOs of IT vendors worth their pay are going to use this economic downturn to revamp business models and position for the so-called post-recession era. Certainly the era of nice-to-have IT projects has gone. The focus is on doing more with less, improving customer loyalty and new revenue opportunities. This is why Progress Software has positioned its business around operational responsiveness, which hits on all these needs. As the economy improves, many businesses are emerging as lean, mean fighting machines - well prepared for growth."

Q. Have you been near any guillotines recently?

I see myself more as the Scarlet Pimpernel - helping my customers avoid the guillotine!!

Economic downturns are scary times for employees of any organisation. Many businesses use these times to restructure their organisations for optimum performance. My advice is to communicate the goals of the company, let employees know they are key to the company's future success and that the organisation is committed to delivering for its team and customers. If employees know that they're an active part of the solution, they can help to identify sources of savings that you may never have thought of.

Q&A with Microsoft's head of environmental sustainability

Read more on IT strategy