On average 40% of activities conducted by application
development teams are associated with application support,
according to a Gartner Benchmarking Report.
Any executive in charge of application development should be
concerned by that figure. And every executive in a software
business should be alarmed.
In its 2004 Benchmark Study, the Service and Support
Professionals Association reported that the percentage of support
cases closed at first contact continues to decline, and the length
of time a case is open continues to increase. Escalation to a
broader resolution team leads to skyrocketing personnel costs.
Longer problem resolution times can cause customer dissatisfaction
and delayed or lost revenues.
These facts are not that surprising.
The software development process (presumably acc- ounting for
the other 60% of a developer’s time) has changed greatly in the
past few decades, but it continues to rely on the same manual,
labour-intensive, iterative approach.
Supporting applications means solving application problems. And
although systems to manage the “trouble ticket process” have been
in place for many years, little progress has been made in
technology to handle the heart of the issue: figuring out the root
cause of the application problem.
The challenge is that the symptoms of a software problem rarely
reflect the root cause. Finding the glitch is not easy when you do
not know where to start looking. A single business transaction may
kick off a sequence of complex processes, each of which may involve
events that happen on a dozen potential servers. The root cause of
the problem could be a software issue, a hardware fault, a
configuration issue or even an end-user’s mistake.
A survey by Market Dynamics found that 75% of the application
problem resolution cycle time is attributed to determining the root
cause of the problem. This can be especially difficult when
problems occur at remote user sites.
Support teams typically go through a lengthy and costly process
that includes conference calls, attempts to gather information,
trips to the user site, and multiple attempts to recreate the
user’s environment and the problem scenario.
Although Gartner research has shown that application faults are
responsible for only 40% of all unplanned downtime, any software
support veteran knows the supplier is guilty until proven innocent.
Clearly, a change in the application support model is long
overdue.
A growing number of software suppliers and in-house teams are
using low-impact application recording technologies. These look
inside a running application to collect the technical information
needed by support specialists for resolving problems.
The most sophisticated recording technologies can capture the
user’s actions preceding a failure, the system configuration,
events, application performance parameters and even the related
code execution flow, and can synchronise this information on a
single timeline. They can provide powerful automated analysis to
dramatically accelerate root cause determination.
In this way the most costly and cumbersome steps of the problem
resolution process are reduced or eliminated. The process of
gathering information from end-users and system administrators at
the customer site drops significantly since the capture of all
necessary information is automated.
Time spent trying to replicate the problem is eliminated, since
the actual problem history has already been captured. (According to
the Market Dynamics report, this problem replication step is
repeated seven times, on average, in the resolution process.) Since
the problem history can simply be replayed, the problem resolution
team can proceed directly to delivery of the fix.
Remote application recording technology has been shown to reduce
problem resolution cycle times by up to 80%, with labour savings of
nearly 60%. Software suppliers that use this technology have seen
dramatic results.
An ERP payables application kept mysteriously crashing on
start-up, but only at one customer site, and on two of six
seemingly identical servers. After investing more than a month and
£35,000 in a multi-engineer analysis, the support team still had
not discovered a cause or cure.
After deploying remote application support technology, the team was
able to solve the problem in less than two hours. Analysis of
problem history logs pinpointed the root cause: access rights to
the required Windows registry key, due to an inadvertent and
undetected change.
A hospital management organisation faced recurring, intermittent
processing delays in a web-based application. The application
supplier, facing stiff service level penalties, deployed a
15-person crisis team which spent some 3,000 hours – costing
£200,000 - without solving the problem.
When the company turned to application support technology, the
first insights came in just a few hours. By isolating the root
cause to specific components in the application server tier, the
supplier was able to return most crisis team members to their
normal responsibilities. Within the next few days, the remaining
engineers pinpointed the problem to threading issues with their Com
object, and solved it.
Had this company deployed such technology from the beginning, it
could have reduced its investment in engineering hours from 3,000
to less than 100, saving more than £175,000 in this incident
alone.
The characteristics that make enterprise applications so
powerful and productive when all goes well makes them nearly
inscrutable when things go wrong. The time has come for a change
in how we support these applications: a technology approach that
can accelerate the problem resolution process and reduce the number
of escalated issues, so the development team can stay focused on
development.
Yochi Slonim is chief executive at Identify Software