Royal Holloway: Testing antivirus efficacy in Linux

Table Of Contents

• We measured the detection rate of antivirus programs installed in virtual machines and available through a well-known on-line malware scanning service.
• We assessed the AVs’ effectiveness over a period of time to find out whether they are affected by regression. When this happens, an antivirus is no longer capable of detecting malware samples that were successfully identified in the past.
  We evaluated the more advanced methods AVs now include to improve the standard signature-based detection mechanism. These methods are known as heuristic and in most cases rely on the identification of behavioural patterns.
• We excluded highly specialised products (for example, Chkrootkit and Rootkit Hunter are well-known scanners, but they are server-oriented and routinely used by systems administrators to detect only specific types of malware) and discontinued products (given that several AV suppliers, such as AVG, Avast, Bitdefender, F-Prot and Zoner, have decided to develop licence-protected products exclusively for Linux servers).
• We tested four locally installed AVs to measure their detection rate and regression effects as well as to assess the efficacy of their update mechanisms. The average detection rate of the tested products was always well above 80% and none of them was affected by regression. Only one of the locally installed AV programs showed a steady increase over time in the number of detected malware samples. We used online malware scanning service VirusTotal to compare the effectiveness of 62 antivirus products. The average detection rate of the online AVs barely reached 60%, but nearly half featured a detection rate above 90%, with one-third showing less than 30%. Interestingly, 13 out of 62 antivirus products showed regression effects.
  We created 24 malicious files by using penetration testing tool Metasploit. They were scanned and then executed to determine the effectiveness of the AVs’ heuristic detection mechanisms. The detection rate was as low as 8.3% and no antivirus program was able to block the execution of samples that had not already been flagged during the initial scan.
  Our customised malware specimens were submitted to VirusTotal as well. The results show that the average detection rate was only 16.9% and that 32 out of 62 AVs did not report as malicious any of the submitted files. These tests show that, while detection rates above 90% are achievable, the signature database update mechanisms should be reviewed and improved. Although only a minority of the tested AVs was affected by regression, the detection rates increased over time very marginally.
• Another area where the considered AV solutions underperformed is heuristic detection, which did not provide any additional layer of protection to the user.