Mutual benefits

Although companies may feel that acting as a testing site for a supplier's new product may be too risky to their business, firms...

Although companies may feel that acting as a testing site for a supplier's new product may be too risky to their business, firms must weigh this up against time they will spend testing for the advantages to be gained from customised software. Liz Warren reports

Why would businesses want to test a supplier's bug-ridden product and even risk implementing it in a live system? Although becoming a beta site may seem a risky choice, many companies that have taken part have experienced many advantages, from customised advancements to their IT systems to savings from personalised software.

One company that has acted as a beta site is Evans Vanadine, a family-run business that manufactures hygiene products and agricultural disinfectants. Evans Vanadine operates from a single site and has a turnover of £14m and 135 employees. The firm has been testing various incarnations of GEAC's manufacturing software for the past 15 years.

"With many systems, you have to change your business to work in the way the software works," says Anthony Evans, IT director at the company. "As a beta site, you can say to the supplier, wouldn't it be nice if the product worked this way? If it agrees, you can tweak it to meet your needs, rather than being lumbered with a fixed product. We also get direct support from the people who wrote the software - not just the support desk."

For the supplier, the ability to garner ideas from customers about how their products should develop is a key benefit of running a beta programme. These can also be used to test specific features that have already been developed.

Ken Drake, director for Northern Europe and Africa at Micromuse, which provides tools for end-to-end management of complex IT and telecoms environments, is particularly interested in working with users who "take a strategic approach to the product and want to help improve it over a long period of time. These customers are constantly looking to extend the product and improve their own implementation to add value. They have the best ideas for where they want the product to develop."

Drake says users who view a product as core to their business will benefit most from becoming part of a beta programme. The programme can give them a chance to increase their level of partnership with the supplier and see their requests for features to meet their organisation's needs put into practice.

John Stanners, support director at messaging software supplier Gordano, is responsible for the company's beta programme. "It is not just about early access to new product releases, but the opportunity to directly influence the supplier's development thinking and create a more wide-ranging and 'special' dialogue with a supplier," he says.

In addition, suppliers may use a single beta site to provide a proof-of-concept environment for a product which has been tested under laboratory conditions, but which needs to be shown to work when faced with user requirements. For instance, Moorfields Eye Hospital has been acting as a pilot site for Intersystems' Ensemble enterprise application integration platform. This collaboration means the hospital has the interface engine it needs to meet its obligations under the NHS' modernisation plan, the national programme for IT, at a fraction of the cost of implementing an established product.

Barrie Winnard, the hospital's IT director, says that under the deal with Intersystems, Moorfields has access to software, implementation and development services from consultancy Stalis. Three interfaces will be created between its patient administration system and three other applications. All of this will be provided for about £20,000, far less than the £180,000 such a project would normally cost.

The tangible benefits offered to beta sites vary enormously from supplier to supplier, but the benefits Moorfields has received are unusual. Stanners says most suppliers are reluctant to offer this kind of benefit as part of their beta programme because some organisations will sign up purely to obtain early versions of software, or in the expectation of free support.

"For this reason, we do not offer beta testing places freely," Stanners says. "We like to see evidence of commitment and a two-way dialogue with our testers, although we will produce incentives in cases where we have evidence of genuine participation and input from a particular tester."

For many user organisations which become beta sites, tangible benefits are less important than the intangible ones. Evans says, "We could happily twiddle our thumbs and let the company drift on, but as a beta site, we see all the new functions and ideas and that spurs us on to make every department implement new thinking."

Nevertheless, Evans does appreciate the discounts GEAC has provided on implementation consultancy and software licences.

A further benefit is the contact that IT teams at beta sites can make with peers in other organisations in the programme. In addition, some beta sites can become reference sites and participate in publicity activities with other firms, but this is not usually a mandatory requirement. Although this can offer good opportunities to make contacts with peers that IT managers might not otherwise encounter, Drake says, "it does take time to prepare properly when hosting a site visit."

The additional time required to respond to the supplier's demands is the main downside of becoming part of a beta programme. The IT team will almost certainly need to get its head round the documentation for the new product and spend time running tests and providing feedback.

Stanners says, "There is an overhead involved in performing a more methodical, thorough and wider ranging series of tests on a product than might be the case if you are simply looking at those features that would be of direct benefit to your organisation. Reporting test results and exploring issues with the supplier can take time."

Drake says that it may involve days or even weeks of additional work and Evans confirms this. "There are many meetings to discuss what is going to be done and how it is going to be done," he says. "Over the past month, for example, I have spent four or five days running through processes for GEAC to make sure it all works."

Although the IT team may find becoming part of a beta programme can create this overhead, disruption to the business itself should be minimal as beta products are usually confined to a test environment rather than run in a live production environment. "We would not expect a beta product to go into the production environment until it is generally available," Drake says.

However, there are situations in which a relatively untested product can only be trialled in a live environment. Organisations involved in those kinds of beta programmes need a contingency plan if the product fails.

Moorfields, for example, will have parallel point-to-point interfaces to which it can revert to if the Ensemble interface engine fails, and Evans Vanadine can survive without its IT systems for short periods of time as the production line can operate even if the computer systems are down.

Clearly, becoming a beta site is not an easy option, but if you are invited to become part of one, you should feel flattered that the supplier has such a high opinion of your organisation. "Micromuse has 1,700 users and only five to 10 will be on each beta programme," Drake says.

Stanners agrees, "We recruit selectively and by invitation. We are looking for a relationship that is open and constructive and we certainly give careful thought before issuing an invitation. We do so in the hope of a positive and mutually beneficial relationship with people who we judge to be technically competent. An organisation should feel flattered to be asked."

Chapman Taylor   

For architectural and design practice Chapman Taylor, getting its hands on the latest beta kit from 3Com was one way to tackle a redesign of its network to accompany a move to new offices.  

 "We had used 3Com equipment before, but USL, our outside cable contractor, introduced us to 3Com on a different scale so we could have a different level of conversation about our new network," says Shan Tilakumara, IT director at Chapman Taylor. 

3Com offered beta versions of products ahead of general release that fitted in with Tilakumara's plans for the network and met the timescale for the office move.  

Because the beta products - 3Com 4050 Gigabit Ethernet switches - were going into a live environment supporting 200 users communicating internally and externally with contractors, Tilakumara was careful to minimise the risks to the business. The switches were run for three months on a test bed and the old kit remained mounted so that Chapman Taylor could revert to proved technology simply by swapping a cable. In the event, the switchover proved very smooth. 

As well as giving it the new network it needed, Chapman Taylor has seen long-term benefits from being involved in the beta programme.  

"We keep in touch with 3Com on a regular basis and talk to them directly about day-to-day problems as well as alert them to features we would like them to integrate into their next generation of products. We have also gained earlier access to other equipment coming through the channel," says Tilakumara.

What to ask before becoming a beta site   

What does your organisation hope to achieve by becoming part of a beta site, rather than working with current, standard releases? Can the supplier deliver any more to you as part of its objectives? 

What will be your organisation's commitment? What will you be expected to sign up to contractually? 

How long will the programme last? 

Will you be expected to take part in future programmes? 

What kind of non-disclosure agreement will you be asked to sign, and what kind of approval will you need from management? 

What kind of testing will you have to undertake over and above your daily tasks and how long will this take? 

What kind of feedback will you be asked to provide, how frequently, and how will it be given - by voice, e-mail, forms or through direct access to the supplier's trouble ticket system? 

Will you expected to take part in external or reference communications that require clearance from your own management?  l Do other aspects of your environment - such as the operating system - match the supplier's test requirements or will you need to do additional work to upgrade your environment? 

What sort of commitment is the supplier offering? 

What kind of documentation will they provide? 

What level of support will you receive?  

How many support staff will have been trained in the beta product and how easily and quickly can issues be addressed? 

Will you be able to speak directly to the product developers? 

How quickly will fixes be provided? 

What tangible incentives, if any, are available, such as lower rates for implementation support? 

How committed is the supplier to making the product work? 

Can you run the tests the supplier requires in a low-risk fashion? If you must move the beta product into a live environment, can you revert quickly and easily to a stable environment?

Read more on IT risk management