Computer system log files are deeply boring and unsexy, but it
is amazing how quickly they can become fascinating when things go
wrong.
These days, there is a growing chance of that happening as
companies try to deal with rising pressure to comply with
regulation. Not only do they need to know what happened when things
go wrong, they also need to satisfy auditors and inspectors that
day to day operations are in compliance.
This is difficult to achieve. Complex computer systems produce
daily millions of data items that record what happened and when.
Printouts of the data take too long and require interpretation by
rare and expensive skilled operators.
Moreover, even the technically gifted are seldom expert in more
than a few systems. Integrating information from different log
files to derive useful information on which to act remains
tricky.
Many companies have tried to solve this problem. One betting its
business on log file analysis is LogLogic, a San Jose company with
a growing list of international offices. Its chief executive, Pat
Sueltz Pat, was president of
salesforce.com, an executive vice-president at Sun
Microsystems, and general manager of the Java software division at
IBM.
According to Sueltz, in the old days, analysing logs was left to
hardcore IT, network and security professionals. But increasingly
those who manage parts of the company, such as finance or human
resources, also need to understand log records, often for
compliance reasons.
She estimates that 30% of an enterprise's data is expressed in
log files. That is a problem. "Very few people these days can read
a core dump and understand what is going on," she says.
LogLogic has developed an appliance, now in its fourth version,
that pulls log data from systems in real-time, and uses software to
graph trends, highlight exceptions and track remote attempts to
access system resources, among other things.
This enables operations staff, administrators and non-technical
employees to work with log files and time-lines to create a
storyboard that captures any chain of events surrounding a given
incident.
Sueltz already has three of the top four telecommunications
network operators as customers, as well as financial services
firms, health care outfits, and government agencies.
The market is presently driven by the need to conform to
regulations, security concerns and the need to run IT more
efficiently because budgets are tightening, Sueltz says. "The logs
are the place to start because they measure everything," she
says.
But that is too much information. Until recently it has been
hard to adjust the flood to what is meaningful. LogLogic has done
this in two ways: one is to graph the data so that it is easier to
identify trends, the other is to report events based on parameters
the client sets that chime with their view of what is a risky
event.
Even so, when the regulations, such as the
Payment Card Industry Data Security Standard (PCI:DSS) mandates
log analysis, that is manna for Sueltz. "The problem with
regulations such as DSS and Sarbanes-Oxley is that so many people
are talking at a high level, and not connecting with operations.
About 25% of companies draw their compliance reports from audit
reports. That is not good enough when things go wrong, and that is
where we can come in," she says.
The economy is giving Sueltz another string to her bow. To get
through the recession, more and more companies are changing the way
they do IT. It is becoming important to establish operation base
lines from which to measure changes. That means logging and
analysing both the current state of the systems and the effect of
changes.
This lets them answer questions about how much the
virtualisation project has actually saved, says Sueltz.
One client, eco-friendly retailer
The Body Shop, has used LogLogic, originally bought to satisfy
PCI:DSS, to identify applications that hog bandwidth. A simple
reconfiguration is now saving bandwidth upgrades.
Some regulations that drive log analysis
Health Insurance
Portability and Accountability Act requires the establishment
of national standards for electronic health care transactions and
national identifiers for providers, health insurance plans, and
employers. It also addresses the security and privacy of health
data. The standards are meant to improve the efficiency and
effectiveness of the nation's health care system by encouraging the
widespread use of electronic data interchange in the US health care
system.
Sarbanes-Oxley.
Sarbanes-Oxley was a response to corporate and accounting scandals
such as
Enron and
WorldCom. The act criminalises non-compliance with tougher
corporate governance standards.
Payment Card
Industry Data Security System (PCI:DSS). A set of security
practices mandated by credit card issuers designed to stop
card-based theft and fraud at the point of sale.
Basel 2. An
international standard that defined how much capital banks need to
put aside to guard against their financial and operational
risks.