Sergey Nivens - Fotolia

Making the most of logs

Most IT systems produce log files, which IT administrators can use to automate security systems

Increasingly, logs can be used to review information security. Unfortunately, complex IT systems have hundreds, if not thousands of different log files, each comprising substantial amounts of data, which may or may not be relevant to the operations and security of the business.

Log management is a content management and big data issue. Everything from Windows to firewalls to servers generates them, which makes collecting and studying the vast amounts incredibly difficult – especially when insights are needed quickly. IT administrators often ignore these automatically generated logs – until the system goes down, when the log files are scrutinised forensically.

According to Mike Gillespie, director of cyber research and security at The Security Institute, logs could be used by the security manager to build security resilience by tweaking or hardening policy or enforcement and measuring the effectiveness of current methodologies; possibly to expand them to other areas. Logs provide a useful way to identify training needs or influence decisions relating to content filtering and appropriate access to internet, social media or areas of the network that should be handled sensitively, he says, and help IT and security chiefs identify time-outs for unattended workstations and allow for increase or decrease in time elapsed as appropriate.

He says: "You can use log data to analyse patterns in behaviour, such as checking to see if users are logging off at the end of their working day or just locking their screens, so any system changes, patches or updates may not take place. Encryption will be rendered useless and the machine is at risk of breach. A pattern of this behaviour or sudden change could be a security warning flag."

Logs for security

Security information event management (SIEM) offers an automated way to tie together all the log data from the network and its security tools, then condense it down into something manageable.

"SIEM tools are a practical way to enable security teams to detect, respond to and prevent incidents in a fast-moving, data-heavy environment. They provide a way to detect anomalies and attacks on a network, by comparing current traffic to the average in real-time. Notifications can then be sent to security personnel to respond and rectify," says Adrian Davis, managing director for Europe at (ISC)2.

Read more on log management

How can log management be used to bolster information security?

Log data needs a clear aim to be useful to security,

Almost everything can be mapped with a log or an event and can be correlated.

SIEM functionality can be extended to automate actions. He says: "If the SIEM detects an abnormally high amount of traffic going out of a PC (a symptom of exfiltration attacks), it can learn this pattern of traffic and automatically stop it if the issue is detected again in the future. This process can be completed much quicker than a human and improves overall security."

A log management and intelligence programme could benefit incident response, moving from reactive to proactive. "I believe in creating a forensic-readiness platform that would leave no system behind, while ensuring that protection and defence is enabled across the whole landscape," says Ramsés Gallego, international vice-president of ISACA, and security strategist and evangelist with Dell Software.

Gallego has experienced projects where certain regulations and legislation have been mapped to the volume of data (or correlated data), to find non-compliance situations. He says: "I have been fortunate enough to work in forward-thinking projects where attacks have been prevented through predictive analysis."

How and when to respond to risk reports

Tim Holma, international board director at the Information Systems Security Association and CEO at 2-se, says: "If we use software to collect and display this information in a meaningful way, analysts can make informed decisions as to the seriousness of a log event in a matter of seconds, and their ability to detect and respond to harmful events improves dramatically."

Holma says log management must be part of the network infrastructure to protect against blended threats: "The way to manage log data lies in the ability to look for user behaviour or attitude changes, plus the ability to monitor activity and report on segregation of duties, dual controls and access violations," he says.

Holma argues that knowing the identity of individuals who access unauthorised data is important, but IT departments and the CISO must ensure the information is correctly organised and correlated to avoid falsely accusing an individual of illegally accessing sensitive data.

Peter Wenham is a committee member of the BCS Security Forum strategic panel and director of information assurance consultancy Trusted Management. He says that, as with any tool, it is in the setup and configuration that success or failure is born. Reports need to be meaningful and generated on a regular but not too infrequent (or frequent) basis. For instance, he says inactive accounts can be reported monthly but the top 10 users should be reported weekly.

"Strange or anomalous behaviour – such as modifying or deleting a system file, detecting malware on a server, unexpected export of a file, heavy use of resource such as CPU or internet bandwidth or multiple user authentication failures, systems unexpectedly going offline – should be defined and issued as an immediate alert," says Wenham.

Another consideration is storage. He says: "Retention time for log/audit data needs consideration as there can be large volumes of data generated and for a typical business a retention time of around three to six months is recommended." All parts of an IT system need to have their clocks synchronised to a single central source – and that synchronised to an external atomic source.

Having good quality alerts issued to an appropriate set of personnel will greatly assist in incident response, as will the availability of time-stamped log and audit reports. By holding log and audit files in a dedicated log server(s) – rather than on each server or device – will assist in any investigation, and making the log server write-only will protect the files from unauthorised access and modification. This will allow forensic investigations to take place, Wenham says. Good quality logs, analysis and reporting will feed into the process of improving security after an incident.

But, says Davis: "Log management and SIEM tools have huge potential to make the lives of security staff easier, but they also have an inevitable impact on user privacy. All devices that generate logs will have an IP or MAC address that is traceable to a user depending on the identity and access management system. Security departments have the ability to go extremely deep into the data, so the practicalities must be balanced with privacy."

Read more on IT governance