Antivirus can introduce dangerous network security holes into any OS

Dangerous flaws abound in most, if not all, applications, but experts say those that hit hardest are found in security products designed to protect you -- in this case antivirus. Research shows that these flaws often require no user interaction -- regardless of operating system -- and privilege level is typically high: system, root or even kernel.

So far this year, independent security researcher Alex Wheeler has uncovered security bugs in Symantec,

Older news on AV flaws Highly critical McAfee flaws patched iDefense reports a Sophos security hole

"It's important to note these AV bugs were reachable by default [normal installations] and required no interaction from the user to exploit them, which is the best scenario for an attacker," said Wheeler, who spoke about his discoveries during this summer's Black Hat Briefings in Las Vegas. "In many of the high-risk bugs being published today in client software -- for example Internet Explorer, iTunes and Mozilla -- the attacker must entice a user to do something in order to exploit the flaw, like view an image, listen to a song or visit a Web page."

AV programs typically run with high privileges, including system, root or even kernel, Wheeler said. This allows an attacker to basically do anything at will, including:

• installing a rootkit to spy on the users of the system and execute programs without being detected;
• exploiting any trusted relationships with other systems to compromise them as well, which is extremely useful when the AV library is running on a gateway system protecting an entire enterprise or ISP network;
• stealing passwords, financial data, etc.; and
• modifying or remove existing data, like an e-mail or document sent to the user.

"These bugs were in core libraries that the vendors used in all their AV products," Wheeler said. Working with Internet Security Systems researcher Neel Mehta, the duo realized the severity of the situation. "So the bugs affected any AV product the vendor was selling, which for larger vendors was sometimes more than 30 different products. Further, these libraries run on pretty much any modern OS: Windows, Linux, Unix, Mac, etc. So it's not just Windows systems that are vulnerable, even though Windows systems account for the majority of virus risk."

Why AV is vulnerable
"AV engines are made so they scan all the data before the user even has a chance to do anything with it, which from an attacker's perspective is exactly what you want," Wheeler said. "You want to be able to trigger a vulnerability without the user having to do anything. These bugs require nothing from the user. It's the antivirus itself that exposes them."

According to Wheeler, the AV engine has to mimic any application that creates files on a computer, like Microsoft Office programs. But it has to enforce more error conditions than the actual product does, identifying bad files and corrupt data. If you put an exploit or any malicious code in a file the virus engine will identify it as a bad file and decide not to scan it any further, but the application will still open it and process it. Wheeler said any differences between how AV handles those sorts of files and how the actual application handles it are potentially exploitable because the exact functionality has to be mimicked, not just something close to it.

"I think it's a natural inclination for AV developers to put more checks in their products than other applications do because they want to be secure. Application developers may not have thought of putting in checks for all those types of conditions," Wheeler said. "The end result is the product will be less strict than the AV engine."

Colliding file formats
And the problem doesn't end there. Wheeler said that because Microsoft Office applications all use slightly different internal structures and storage areas for file formats, an attacker might be able to make such a file also look like an executable. If both formats can be matched, the AV engine has to make a decision on which format to scan first -- something he says AV vendors haven't planned. An attacker could use this technique to evade detection.

"Using a security product is supposed to protect you -- not hurt you -- so having flaws in it kind of defeats the purpose," Wheeler concluded. "I think we're just scratching the surface with AV flaws right now."