Log management technology has developed very rapidly over the past few years, largely due to the demands of audit and compliance regulations such as the PCI Data Security Standard and other government guidelines. These compliance obligations have produced a plethora of new log management and/or security information and event management products from established players such as RSA Security Inc. and IBM, as well as new emerging companies such as LogLogic Inc. and LogRhythm Inc.
The variety of available tools can make choosing a suitable log management product a daunting and difficult task. Most of the solutions claim to meet organisations' compliance and data management needs, making selection confusing. With this in mind, here are some considerations to short-cut the process and find the best log management product for your specific enterprise:
Log management or security event management
Log management and security information and event management (SIEM) are not the same, so it's vital to decide which function suits your business needs. Log management technology is exactly what it says -- it records information from a variety of sources such as firewalls, switches and host systems. SIEM tools log information, but provide the resulting data in context. For example, a SIEM product may highlight repeated login failures on a particular server, marking it as an event that needs investigation.
Whilst many vendors offer log management and SIEM, look carefully at how each one performs individually and don't compromise on key features and functionality, especially if you're considering a combined log management and SIEM tool. The convenience and time savings offered by a single box can be attractive, but unless both the log management and SIEM functions are truly effective, it's a false economy. The ability to simply store and archive logs is not particularly useful in itself, but the intelligence and data enrichment which turns logs into events is essential.
Define event correlation for your business
It is important to define the information that you need about an event and use it to benchmark possible products. Look for solutions that offer a high level of correlation between events. Without this, you have to trawl through vast amounts of data manually to correlate incidents, which is difficult and time-consuming. Look for tools that correlate scans from different security and network devices. Also, a solution that drills down into the event's specifics is far more useful than one that simply tells you "something" has happened.
Check the support for data collection sources very carefully
Carefully consider how a product supports real-time collection and analysis of log data. Make sure it supports the range of devices from which you need to gather data and that the vendor can incorporate non-standard/unsupported devices without extensive customisation. It's vital to find out exactly which devices and systems vendors will and will not support. Solutions can be customised, but the cost and time involved may make them prohibitively expensive.
Data storage and access must be straightforward
By their very nature, log management tools must store data over long periods. Typically for PCI DSS compliance, logs need to be stored and accessible for 6 to 12 months. At the same time, being able to get an instant view of the last 24 hours in a 'dashboard' format is extremely useful. Consider how easily the different products can archive, store and access vast amounts of data, as well as how effectively you can compare stored data over different time periods. If it can't be done quickly and simply, your proposed log management product may be unworkable without the manpower or resources to support it.
Detailed reporting is vital
When looking for the best log management tool, source a product with excellent "off the shelf" reporting as this feature saves time, cost and doesn't overburden existing resources. Consider the amount of work necessary to produce the analytical data that your organisation needs. For example, how easily could a report be run to show all the events generated by a typical user during a day, across all devices? Some products produce huge amounts of data, but if it's only useful after lengthy analysis and interpretation, it's probably not the best option for a small, overstretched IT team.
For many organisations, tools which use preconfigured reports, dashboards and graphics are an excellent option -- just ensure they can display the information that's most relevant to your business. Make sure too, that you can quickly and easily drill down from an event to the raw log data so that you can see exactly what happened. If this is difficult, it could be a costly and time-consuming process every time you need to look at the details.
There is no doubt that log management/SIEM is a maturing technology; the market will see many more new entrants, each offering another option or enhanced features. However crowded the market becomes, these simple steps offer a starting point to make an informed decision.