Rooting out a rootkit: Stage one -- Diagnosis

Antivirus software has been mysteriously disabled, some users get the blue screen of death and others have experienced unexplained restarts or error messages. Find out if these problems are caused by a rootkit.

Given the information in the scenario, is a rootkit to blame? Read what the experts have to say, or click here to go back to the scenario.

Kurt Dillard: The details are lacking, but a few key pieces of information stand out. Frequent operating system crashes on a variety of systems that had been reliable means that something has changed on the affected computers. Antivirus software automagically disabling itself is another significant clue. Finally, the fact that the standard security tools can't find any malware suggests that if there is new software on these computers, it is running in stealth mode with files hidden from view but still operating.

If the only odd thing is numerous system crashes, I suspect a recent patch to the operating system, device drivers or a security application. But the combination of symptoms implies something nefarious is in play. Nevertheless, it may not be a rootkit. You'll have to do some additional research to figure out what's going on.

Lawrence Abrams: When in a situation where abnormal behavior starts occurring on your computer, the first thing that comes to mind is that you are infected with a piece of spyware, a virus, Trojan, worm or other type of malware.

If you continue to have problems after scanning your computer with antivirus and/or antispyware software, it's then time to use some tools that provide deeper insight. What need to be examined are the computer's start-up programs to see if there is a new malware currently not in the antivirus definitions. Some programs that detect troublemakers are:

  • HijackThis: general homepage hijacker detector and remover, continually updated.
  • WinPFind: scans common locations on the hard drive for files that match patterns known to be used by malware.
  • Silent Runners: pinpoints how Windows starts up and creates a text file for study or to be stored as a benchmark.

If nothing is detected, try running the programs and your antivirus/antispyware in safe mode. Many of the common generic rootkits being released with worms do not run in safe mode, so safe mode makes them visible to the troubleshooting software.

If new entries and files are found in safe mode, than the computers are most likely infected with a generic rootkit that can't be seen in normal Windows mode. On the other hand, if after you run the same programs in safe mode and there is still nothing suspicious, yet the behavior continues, you can assume that you are dealing with a more targeted rootkit.

Kevin Beaver: Considering the strange behavior of the installed applications, odds are you're dealing with some type of malware -- most likely a rootkit or a remote access Trojan, which may be allowing surreptitious access from offsite to an unprotected machine. The only way to know is to run additional scanning software that can scan or monitor for anomalous behavior and rootkit presence. Such tools would be Sana Security Inc.'s Primary Response, the various solutions offered by Finjan Software LTD and Sysinternals' RootkitRevealer.

I'd still recommend running at least two or three additional antispyware programs as well. Odds are there are still a handful of utilities you may not have tried. Look beyond the common solutions of Spybot--Search & Destroy and Lavasoft's Ad-Aware. I've had good luck with Computer Associates' PestPatrol and Microsoft's AntiSpyware. Two other good tools for monitoring for odd system activity are personal firewall programs that monitor and/or block outbound traffic (not Windows Firewall) as well as a network analyzer that can monitor network traffic to and from the suspect system. Of course, this latter option is good only if you leave the system connected to the network.

Stage two: Immediate actions

About the experts: Expert bios are available on the scenario page.

Read more on IT for small and medium-sized enterprises (SME)