For an attacker to compromise a system, he or she must tamper with the computer's system files in some way, especially to maintain continued access and prevent detection. This may involve deleting particular files, altering or replacing them, or even adding files.
A file integrity checker shouldn't replace an intrusion detection system, but should work alongside it, alerting you when an intruder has slipped past your IDS and begun to compromise your system.
The best way to learn how to detect hacking and monitor system files for tampering is with a file integrity checker. A file integrity checker calculates a hash value, usually MD5 or SHA-1, of important files and stores them in a database. The checker can then be used to calculate and compare the current hash values of these files against those in the database. If any values don't match exactly, the file has been changed in some way: its content, size or permission settings for example. This process not only alerts security pros to a potential system compromise, but also indicates which files have been used in the attack and, based on the number of conflicting checksums, the extent to which the system has been compromised. This can make forensic investigation and system restoration a lot easier than trying to work blind if system diagnostic tools have all been replaced by a rootkit.
The first step in the file integrity-checking process is to create a reference database while the system is in pristine condition so you know exactly what your file system is supposed to look like. This means the check must be run before the database server is connected to any other computer or network and that all the software installed on it has been obtained from reputable or validated sources. Otherwise you could be creating hashes of a system that is already compromised, giving you a false and dangerous sense of security. Make sure also to store the reference database offline, or an attacker may be able to compromise the system and hide his or her tracks by modifying the reference database.
Once you have your reference database, run regular checks -- daily, or even hourly if the system is a high-profile target -- to compare the current state of the file system with the reference database. If any unexpected files show up or hash values no longer match, then there's a possibility the system has been breached.
Obviously, when updating programs or applying patches, be sure to update the reference database as well. (Actually running a check immediately after any sort of system update will show you what files have been modified or added, which can help troubleshoot any configuration or compatibility problems.)
The best known product for automating file integrity checking is that made by Tripwire Inc. Although originally an open source project, Tripwire is now a commercial product. Alternatively, there is an open source version of Tripwire, and although it can check and monitor Windows systems, the actual program only runs on Unix/Linux-like operating systems. Other open source file integrity checkers include AIDE, OSSEC and Samhain, but these too are built to run on Unix/Linux systems.
So is there any open source tool available for Windows-based networks and administrators who don't like working at the command prompt?
Afick (Another File Integrity Checker) is based on the Tripwire tool and has been designed to work on all platforms, including Mac OS X, Unix, Windows and Linux. To run it as a Microsoft file integrity checker on a Windows machine, you only need to have Perl and a few standard modules installed. I find the easiest way to get it up and running is to install ActivePerl Community Edition and use its Perl Package Manager to install the Tk module which is needed to run afick-gui, the graphical interface to the command-line tool. There is also afick-webmin, which allows you to administer afick remotely via a Web browser.
The great thing about afick is that it can load any number of configurations, so you can run a quick check just against key files every hour to monitor for unexpected changes, and load a more comprehensive check if you suspect you've been hacked. As I've already mentioned, the database used to store your reference database should be written on read-only media, so an attacker can't amend or erase it in order to hide his or her changes. The program itself should be run from a well-protected machine or, better still, a Live-CD, so that you can be confident in the reliability of the results.
A file integrity checker shouldn't replace an intrusion detection system (IDS), but should work alongside it, alerting you when an intruder has slipped past your IDS and begun to compromise your system. IDSes typically search for signatures of known exploits, so they're certainly not foolproof when it comes to new exploits, but most exploits will need to make changes to the file system.
By letting you know exactly what's been changed on your system, a file integrity checker provides an extra pair of eyes. If you do get hacked, it helps you to know which programs you can trust and which may be infected and are now malicious. Using the free, open source tool afick, or one of the other open source checkers, is a great way to add another layer of defence to your network.
About the author:
Michael Cobb, CISSP-ISSAP is the founder and managing director of Cobweb Applications Ltd., a consultancy that offers IT training and support in data security and analysis. He co-authored the book IIS Security and has written numerous technical articles for leading IT publications. Mike is the guest instructor for several SearchSecurity.com Security Schools and, as a SearchSecurity.com site expert, answers user questions on application security and platform security.
This was first published in November 2010