Address false positives effectively to avoid cases of mistaken identity

Reputation technology can be used to address the problem of false positives, which has become a challenge for information security professionals.

Today, the security industry is trying hard to keep up with ever-evolving cyber criminal tactics and explosion of malicious software. However, it's not just the bad guys who are keeping infosec engineers on their toes. False positives—the erroneous identification of clean files as malware—have become a challenge too.

Similar to an innocent person being found guilty of a crime he did not commit, false positives identify legitimate files and applications as malicious software. This causes not just a huge resource drain, but also system downtime, data loss and eventual lack of trust in security software.
False positives essentially stem from the traditional reactive approach to security— the blacklisting model. Years ago, when threats were a few large-scale attacks aimed at generating headlines, this approach helped to prevent mass infection. However, there has been a dramatic shift since then.

Security vendors have been churning out signatures at such a high rate in response to increasing volumes of malicious code threats that the scalability of blacklisting paradigm is in question.

Today, the most dangerous attacks compromise a few systems to steal confidential information. These threats are polymorphic, which means that they can easily hide because nearly every instance is slightly different from its predecessor. In fact, 1.6 million new malicious signatures were created in 2008, equating to over 60% of the total signatures ever created. This means that security vendors are creating signatures for millions of pieces of malware which may affect just one or a few systems in the world. Further, the industry has reached an inflection point where there is more bad code created than good code, rendering ineffective the traditional models of security—the mainstay of malware detection for several years.

Continuously increasing the volume of malicious code is essentially a tactic used by malware authors to stretch the capabilities of security vendors. The goal is to get malicious code installed on a target system to exploit the resources associated with that system. Security vendors have been churning out signatures at such a high rate in response to these threats that the scalability of the blacklisting paradigm has been called into question.

The release of high volumes of signatures by antivirus vendors has also led to concern over false positives due to the high market penetration and install base of antivirus technology. Signatures can make mistakes and report vulnerabilities which may not exist, misreport attack attempts, or report vulnerabilities to products that the customer may not run.

This means a new model of security is required, one that does not rely solely on signatures. For example, reputation technology is far ahead of traditional approaches such as blacklisting and heuristics, creating an extra layer of protection against threats that these models are not likely to detect. Reputation technology can also address the problem of false positives, since it evaluates every file based on its reputation rather than signature. This reputation score is determined using complex algorithms that combine various file attributes. As a file is distributed across the Internet and these attributes change, the reputation is updated. This model leverages data from multiple sources, including users and software publishers. Reputation is especially important when a file is new, likely to be a threat, and traditional defenses are not likely to detect it. It defeats an attacker's ability to mutate malware to evade traditional signature-based detection. In fact, the more an attacker modifies a threat, the more obvious it will be that the file is suspicious.

Reputation technology, when used in combination with existing protection models, provides far more accurate detection and significantly reduces false positives. Apart from providing an additional layer of protection, reputation technology allows existing security technologies, including heuristics and behavior-based detection, to be deployed more aggressively to increase the overall level of protection.

About the author: Shantanu Ghosh is the vice president of India product operations at Symantec. He spearheads Symantec's India Innovation centres in Pune and Chennai. In his previous role, Shantanu headed the Security and Data Management group at Symantec India.

Read more on Hackers and cybercrime prevention