How effective are gigabit intrusion detection systems?

In the first of a series of articles looking at the strengths and weaknesses of network technologies, Bob Walder, director of...

In the first of a series of articles looking at the strengths and weaknesses of network technologies, Bob Walder, director of testing company NSS, looks at gigabit intrusion detection systems

In a recent survey commissioned by VanDyke Software, 66% of the companies questioned said they perceive system penetration to be the biggest threat to their business.

The survey revealed the top eight threats experienced by companies to be:

  • Viruses (78%)
  • System penetration (50%)
  • Denial of service attacks (40%)
  • Insider abuse (29%)
  • Spoofing (28%)
  • Data/network sabotage (20%)
  • Unauthorised staff access (16%).

Although 86% of respondents use firewalls - a disturbingly low figure - it is clear that firewalls are not always effective against intrusion attempts. The average firewall is designed to deny clearly suspicious traffic - such as an attempt to telnet to a device when corporate security policy forbids telnet access. But is also designed to allow some traffic through - web traffic to an internal web server, for example.

The problem is that many exploits attempt to take advantage of weaknesses in the protocols that are allowed through perimeter firewalls, and once the web server has been compromised, this can often be used as a springboard to launch additional attacks on other internal servers. Once a "rootkit" or "back door" has been installed on a server, the hacker will have unfettered access to that machine at any point in the future.

Business case

The business case for installing intrusion detection systems (IDS) has never been clearer. The computer world's equivalent to the burglar alarm, an IDS provides valuable back-up to the beleaguered firewall system.

As in the physical world, this "burglar alarm" provides valuable notification that someone has managed to breach perimeter security measures. It should allow IT staff to determine exactly what happened during the attack, and hopefully provide indications of how the security weakness might be addressed.

In some circumstances, the IDS can even retaliate by tearing down the TCP connection being used by the attacker before any serious damage is done. This is the equivalent of picking the intruder up by the scruff of the neck and drop-kicking him out of the door.

When discussing IDS in this article we will be concentrating exclusively on network IDS (NIDS), rather than host-based IDS (HIDS). The latter relies on agents installed on each host to be protected in order to watch for such trigger conditions as changes in file permissions, unauthorised or suspicious log file entries, or alterations to critical system files.

The NIDS, on the other hand, monitors traffic on the wire in real time, examining packets in detail in order to detect patterns of misuse - perhaps spotting denial of service attacks or dangerous payloads - before the packets reach their destination and do the damage. They do this by matching one or more packets against a database of known "attack signatures", or performing protocol decodes to detect anomalies, or both. These signature databases are updated regularly by system suppliers as new attacks are discovered.


Most of the network-based IDS currently available work in what is known as "promiscuous mode". This means they examine every packet on the local segment, whether or not those packets are destined for the IDS machine (much like a network monitor, such as Sniffer). Given that they have a lot of work to do in examining every packet and tracking active sessions, they usually require a dedicated host on which to run.

One of the biggest problems faced by NIDS products in recent times has been the rapid increase in affordable bandwidth. When IDS products first appeared, 10mbps was the fastest network around. Very quickly, however, network speeds have increased to 100mbps, and now 1gbps and beyond. And each increase in network speed seems to take less time to become affordable.

Unfortunately, IDS have had a tough time keeping up, and tests run over the past two years have highlighted several products that had serious problems acquiring even 100Mbit of traffic without dropping packets. Gigabit has proved a real challenge - so much so that even though some IDS appliances sport 1Gbit interfaces, the suppliers are only rating them at multi-100Mbit speeds.

Dropped packets are the IDS designers biggest nightmare, since if even one of those dropped packets is one of those used in the exploit data stream, it is possible that the entire exploit could be missed. It has rapidly become apparent that in order to achieve 100% detection rates at all network loads using all packets sizes, some custom hardware is required.

Today's network cards - and especially the default drivers that come with them and the default packet capture mechanisms in most operating systems - are simply unable to handle a full gigabit of traffic at wire speed in promiscuous mode. High-performance network cards with custom drivers or custom, application-specific integrated circuits are the only real solution - anything less is unlikely to be able to handle extreme loads.

Of course, this does not necessarily mean that a gigabit IDS appliance that uses standard gigabit network cards is useless. Far from it - when it comes to custom hardware, the main aim is to ensure that the appliance can handle the most extreme traffic loads, or even multiple gigabit segments in a single device.

However, the majority of network administrators currently deploying gigabit networks will find that they are seeing far less than a full 100Mbps of traffic moving over the wire, and for these implementations even a standard PC with an off-the-shelf gigabit network card will be able to cope quite easily.

It is vitally important for the administrator to be able to characterise the traffic on the network as accurately as possible. They must ask:

  • What is the average bandwidth?
  • What are the peaks (remember, even one dropped packet during peak activity could allow an exploit to slip through unnoticed)?
  • Is the traffic mainly one protocol (there are so many HTTP exploits that need to be covered by the IDS that some IDS engines struggle on networks that are predominantly HTTP traffic) or a mix?
  • What is the average packet size and level of new connections established every second - both critical parameters that can have detrimental effects on some IDS engines, especially if the packet size is small and the connections per second figure is high?

Deployment issues too become more complex as network speeds increase. Most 10mbps networks used shared hubs, and it was a simple matter to install a promiscuous-mode IDS device on the hub in order to see all the traffic on the network. Gigabit networks require a pure switched environment, which means that if the IDS is attached to a switch port it will only see the traffic on its own port - not really much use.

Deployment issues

One solution is to connect the IDS to a Span or mirror port on the switch, to which is copied all the traffic from all the other ports. Now, however, we revisit an earlier problem - what if the switch is the consolidation point for several other switched networks within the corporate network?

You might think that would be the ideal point to place an IDS, since it would ensure that all traffic is seen. But if the administrator is mirroring just four segments and each segment has just 300mbps of traffic - well within the capabilities of almost any gigabit IDS product - the total traffic being mirrored on a single 1,000mbps port is 1,200mbps. Something has to give, and the result is more dropped packets.

Even where traffic levels are kept within 1,000mbps and the IDS engine is geared up to handle it, some switches are unable to mirror that amount of traffic effectively - once again the result is dropped packets, but now it is the fault of the switch.

Other possible solutions involve the use of network taps and load-balancing switches, but the fact remains that it is critically important to match the traffic levels presented to the IDS to the capabilities of both the IDS sensor itself and the switch on which it is installed.

In some respects, detection performance is the least of the problems facing the administrator tasked with deploying these devices. The problem with any gigabit IDS product is, by its very nature and capabilities, the amount of alert data it is likely to generate. With 1gbps of traffic passing through the IDS, the number of alerts could reasonably be expected to be 10 times that generated by the typical 100mbps product.

Overburdening staff

How many members of staff would be needed to process, investigate and resolve that number of alerts? And how long before the IDS becomes just another device in the corner that is largely ignored thanks to its insistence on overloading the administrator on a daily basis?

It is vitally important that the signature set is tuned to match the traffic on the network being protected in order to eliminate as many false positives as possible. Automated centralised correlation systems can also be of huge benefit when it comes to sifting through masses of data from multiple IDS sensors to identify the most urgent threats.

There is a clear case for implementing an IDS, but network managers must be aware of the potential pitfalls to be avoided when deploying gigabit IDS products.

As with most network technologies, the higher the speeds, the more complicated the deployment. Security is no exception, but if approached with sufficient thought, care and planning, gigabit IDS can be a highly effective weapon against a wide range of network intruders.

Pushing gigabit intrusion detection systems to the limit   

As part of a recent group test project, the NSS Group tested the market-leading gigabit IDS products: Cisco IDS-4250-XL V4.0, Internet Security Systems Realsecure 7.0 Gigabit Sensor, Intrusion Securenet 7145C V4.3, Intruvert Intrushield 4000 V1.2 and Symantec Manhunt V2.11. 

Testing reliably at gigabit speeds presented new challenges and pushed testing equipment to the limits. At the end of the day, the environment created was one of the toughest ever likely to be faced by the average IDS product, while remaining fair in that it represented an extremely busy "normal" network in terms of background traffic. 

Pushing the products under test to their limits in a heavily utilised gigabit network produced some interesting results, and posed problems for some suppliers. What was surprising, however, was that every one of the products turned in what NSS would consider to be good performances in terms of both signature recognition and detection rates.  

In real-world tests, not one product dropped below the 600mbps mark with zero packet loss, and even those products that dropped into the 600mbps+ range demonstrated reasonably high detection rates at higher loads.  

Two products claimed to be able to handle more than 1gbps of traffic - Intrushield and Manhunt - and Intrushield was alone in being presented as a hardware appliance using totally customised application-specific integrated circuits, thus providing extremely high levels of performance at all network load levels and under all traffic conditions. 

Another feature that has improved in recent months is IDS' management and reporting capability. Most of the leading suppliers have redeveloped, or are in the process of redeveloping, their management consoles, and in all cases the improvements are significant.   Key highlights are Manhunt, with an automatic correlation feature taking a lot of the work of identifying related incidents away from the administrator; the advanced data mining and centralised policy management capabilities of Securenet; correlation and alert handling within ISS Siteprotector; and the centralised policy management and alert handling capabilities of both Cisco and Intrushield. 

None of the management consoles are perfect, though it is interesting to see them all adopting similar features and methods of working.

About the NSS Group   

Bob Walder, a leading authority on network security, is one of the founders of the NSS Group and author of the NSS report Gigabit IDS Group Test (Edition 1), which is available  at 

The NSS Group is Europe's foremost independent security testing facility. Based in the UK, with separate security and network infrastructure testing facilities in the South of France, the NSS Group offers a range of specialist IT, networking and security-related services to suppliers and end-user organisations throughout Europe and the US.   

Output from the labs, including detailed research reports, articles and white papers on the latest network and security technologies, are made available on the NSS website at

Read more on IT operations management and IT support