Although companies may feel that acting as a testing site for a
supplier's new product may be too risky to their business, firms
must weigh this up against time they will spend testing for the
advantages to be gained from customised software. Liz Warren
reports
Why would businesses want to test a supplier's bug-ridden product
and even risk implementing it in a live system? Although becoming a
beta site may seem a risky choice, many companies that have taken
part have experienced many advantages, from customised advancements
to their IT systems to savings from personalised software.
One company that has acted as a beta site is Evans Vanadine, a
family-run business that manufactures hygiene products and
agricultural disinfectants. Evans Vanadine operates from a single
site and has a turnover of £14m and 135 employees. The firm has
been testing various incarnations of GEAC's manufacturing software
for the past 15 years.
"With many systems, you have to change your business to work in the
way the software works," says Anthony Evans, IT director at the
company. "As a beta site, you can say to the supplier, wouldn't it
be nice if the product worked this way? If it agrees, you can tweak
it to meet your needs, rather than being lumbered with a fixed
product. We also get direct support from the people who wrote the
software - not just the support desk."
For the supplier, the ability to garner ideas from customers about
how their products should develop is a key benefit of running a
beta programme. These can also be used to test specific features
that have already been developed.
Ken Drake, director for Northern Europe and Africa at Micromuse,
which provides tools for end-to-end management of complex IT and
telecoms environments, is particularly interested in working with
users who "take a strategic approach to the product and want to
help improve it over a long period of time. These customers are
constantly looking to extend the product and improve their own
implementation to add value. They have the best ideas for where
they want the product to develop."
Drake says users who view a product as core to their business will
benefit most from becoming part of a beta programme. The programme
can give them a chance to increase their level of partnership with
the supplier and see their requests for features to meet their
organisation's needs put into practice.
John Stanners, support director at messaging software supplier
Gordano, is responsible for the company's beta programme. "It is
not just about early access to new product releases, but the
opportunity to directly influence the supplier's development
thinking and create a more wide-ranging and 'special' dialogue with
a supplier," he says.
In addition, suppliers may use a single beta site to provide a
proof-of-concept environment for a product which has been tested
under laboratory conditions, but which needs to be shown to work
when faced with user requirements. For instance, Moorfields Eye
Hospital has been acting as a pilot site for Intersystems' Ensemble
enterprise application integration platform. This collaboration
means the hospital has the interface engine it needs to meet its
obligations under the NHS' modernisation plan, the national
programme for IT, at a fraction of the cost of implementing an
established product.
Barrie Winnard, the hospital's IT director, says that under the
deal with Intersystems, Moorfields has access to software,
implementation and development services from consultancy Stalis.
Three interfaces will be created between its patient administration
system and three other applications. All of this will be provided
for about £20,000, far less than the £180,000 such a project would
normally cost.
The tangible benefits offered to beta sites vary enormously from
supplier to supplier, but the benefits Moorfields has received are
unusual. Stanners says most suppliers are reluctant to offer this
kind of benefit as part of their beta programme because some
organisations will sign up purely to obtain early versions of
software, or in the expectation of free support.
"For this reason, we do not offer beta testing places freely,"
Stanners says. "We like to see evidence of commitment and a two-way
dialogue with our testers, although we will produce incentives in
cases where we have evidence of genuine participation and input
from a particular tester."
For many user organisations which become beta sites, tangible
benefits are less important than the intangible ones. Evans says,
"We could happily twiddle our thumbs and let the company drift on,
but as a beta site, we see all the new functions and ideas and that
spurs us on to make every department implement new thinking."
Nevertheless, Evans does appreciate the discounts GEAC has provided
on implementation consultancy and software licences.
A further benefit is the contact that IT teams at beta sites can
make with peers in other organisations in the programme. In
addition, some beta sites can become reference sites and
participate in publicity activities with other firms, but this is
not usually a mandatory requirement. Although this can offer good
opportunities to make contacts with peers that IT managers might
not otherwise encounter, Drake says, "it does take time to prepare
properly when hosting a site visit."
The additional time required to respond to the supplier's demands
is the main downside of becoming part of a beta programme. The IT
team will almost certainly need to get its head round the
documentation for the new product and spend time running tests and
providing feedback.
Stanners says, "There is an overhead involved in performing a more
methodical, thorough and wider ranging series of tests on a product
than might be the case if you are simply looking at those features
that would be of direct benefit to your organisation. Reporting
test results and exploring issues with the supplier can take
time."
Drake says that it may involve days or even weeks of additional
work and Evans confirms this. "There are many meetings to discuss
what is going to be done and how it is going to be done," he says.
"Over the past month, for example, I have spent four or five days
running through processes for GEAC to make sure it all
works."
Although the IT team may find becoming part of a beta programme can
create this overhead, disruption to the business itself should be
minimal as beta products are usually confined to a test environment
rather than run in a live production environment. "We would not
expect a beta product to go into the production environment until
it is generally available," Drake says.
However, there are situations in which a relatively untested
product can only be trialled in a live environment. Organisations
involved in those kinds of beta programmes need a contingency plan
if the product fails.
Moorfields, for example, will have parallel point-to-point
interfaces to which it can revert to if the Ensemble interface
engine fails, and Evans Vanadine can survive without its IT systems
for short periods of time as the production line can operate even
if the computer systems are down.
Clearly, becoming a beta site is not an easy option, but if you are
invited to become part of one, you should feel flattered that the
supplier has such a high opinion of your organisation. "Micromuse
has 1,700 users and only five to 10 will be on each beta
programme," Drake says.
Stanners agrees, "We recruit selectively and by invitation. We are
looking for a relationship that is open and constructive and we
certainly give careful thought before issuing an invitation. We do
so in the hope of a positive and mutually beneficial relationship
with people who we judge to be technically competent. An
organisation should feel flattered to be asked."
Chapman Taylor
For architectural and design practice Chapman Taylor, getting
its hands on the latest beta kit from 3Com was one way to tackle a
redesign of its network to accompany a move to new offices.
"We had used 3Com equipment before, but USL, our outside cable
contractor, introduced us to 3Com on a different scale so we could
have a different level of conversation about our new network," says
Shan Tilakumara, IT director at Chapman Taylor.
3Com offered beta versions of products ahead of general release
that fitted in with Tilakumara's plans for the network and met the
timescale for the office move.
Because the beta products - 3Com 4050 Gigabit Ethernet switches
- were going into a live environment supporting 200 users
communicating internally and externally with contractors,
Tilakumara was careful to minimise the risks to the business. The
switches were run for three months on a test bed and the old kit
remained mounted so that Chapman Taylor could revert to proved
technology simply by swapping a cable. In the event, the switchover
proved very smooth.
As well as giving it the new network it needed, Chapman Taylor
has seen long-term benefits from being involved in the beta
programme.
"We keep in touch with 3Com on a regular basis and talk to them
directly about day-to-day problems as well as alert them to
features we would like them to integrate into their next generation
of products. We have also gained earlier access to other equipment
coming through the channel," says Tilakumara.
What to ask before becoming a beta site
What does your organisation hope to achieve by becoming part of
a beta site, rather than working with current, standard releases?
Can the supplier deliver any more to you as part of its
objectives?
What will be your organisation's commitment? What will you be
expected to sign up to contractually?
How long will the programme last?
Will you be expected to take part in future programmes?
What kind of non-disclosure agreement will you be asked to sign,
and what kind of approval will you need from management?
What kind of testing will you have to undertake over and above
your daily tasks and how long will this take?
What kind of feedback will you be asked to provide, how
frequently, and how will it be given - by voice, e-mail, forms or
through direct access to the supplier's trouble ticket system?
Will you expected to take part in external or reference
communications that require clearance from your own management? l
Do other aspects of your environment - such as the operating system
- match the supplier's test requirements or will you need to do
additional work to upgrade your environment?
What sort of commitment is the supplier offering?
What kind of documentation will they provide?
What level of support will you receive?
How many support staff will have been trained in the beta
product and how easily and quickly can issues be addressed?
Will you be able to speak directly to the product
developers?
How quickly will fixes be provided?
What tangible incentives, if any, are available, such as lower
rates for implementation support?
How committed is the supplier to making the product work?
Can you run the tests the supplier requires in a low-risk
fashion? If you must move the beta product into a live environment,
can you revert quickly and easily to a stable environment?