You may have taken the virtualisation shilling, but is your network up to the job? With storage and server operating systems both being reinvented as virtual entities, companies must ensure that networks originally designed for relaying traffic between physical boxes are able to cope with the unique demands of a virtualised environment.
Colin Butcher, technical director of IT consulting firm XDelta, reiterates the basics of system design. Data is entered, processed, and stored. "The things that matter to most people are availability and predictability," he says (does your system respond, and does it respond the same way repeatedly?).
The elements underpinning that are bandwidth, and latency. Fail to get the mix right, and you'll experience 'jitter' (when packets get dropped through poor latency). When you virtualise storage and servers, how can you guarantee adequate availability and predictability?
"You can have contention on the network during periods of high congestion thanks to too many virtual instances," Butcher warns. A common mistake leading to contention is the under-provision of physical network resources to support virtualised ones. If you're hanging a hundred virtual machines off of one relatively small physical link, the chances are that they're going to run into problems.
"A good example are blade machines," says David Black, a distinguished engineer in the office of the CTO at EMC. "You might get limited to two network ports or SAN ports, for example. So blades sound great in a virtualised environment, but until the advent of converged ethernet, they may not be as useful as you think."
This can be a particular problem in storage area networks, where storage arrays often have to work at very low latencies. "Having sub-millisecond response times is often critical for the performance of the application," says Andrew White, a senior consultant at IT consultancy Glasshouse Technologies. Get it wrong, and the ramifications could be significant.
"Your cluster could decide that a failure has happened. The heartbeat might have a 15 millisecond latency limit, and it could cause VMware to start a virtual machine somewhere else on the cluster," says White.
Choosing which network protocol to use for a SAN will also be important when managing performance. Conventionally, technologies such as Infiniband or Fibrechannel (which operates at layer 2 of the OSI stack) were used. Fibrechannel's advantage was speed, thanks in part to its relatively high bandwidth when first ratified, and to its 'cut through' technique, in which switches begin sending packets onward before the whole packet has entered the port.
Conventional IP networking uses 'store and forward' techniques, holding onto a packet until it has been fully received before sending it on. iSCSI is now being used more readily by White's customers, though.
Whichever protocol you choose, Butcher advocates SAN partitioning based on application functionality. "You probably don't want to mix critical life and death stuff with email. You probably want them on different storage arrays and network fabrics to minimise the risk of one system hitting another," he says.
Typically, you'll use a virtual fabric (or VSAN) to segment your storage area network. In this scenario, N_Port ID Virtualisation (NPIV) can be useful. It enables multiple fibre channel initiators to share a single physical port, virtualising the fibre channel connection. This can make it easier when connecting multiple virtualised servers to a fibrechannel port.
Similar concepts apply when creating networks to support virtual servers, says Scott Herold, chief architect at Vizioncore, which sells software to optimise virtualised systems.
VMware uses the concept of virtual switches in its ESX server, he explains. Virtual switches are normally connected to a physical network interface card (NIC) so that they can get to the outside world, but you'll sometimes want to avoid that connection if you're dealing with a virtual network designed purely for internal management purposes.
You can also assign VLAN tags to hypervisors, so that they can route virtual machine traffic through specific NICs. "I could have two ethernet adaptors with five different VLANs published to the hypervisor," he says.
All of this creates opportunities for efficiency, but it places more pressure on the person tasked with managing the virtualised system.
"If you're changing cables around and something doesn't work, it's not like in a physical environment where you're taking down just one or two servers," says Jason Richards, senior technical consultant at virtualisation consulting company Panacea Services.
Adequate documentation is vital if you're going to manage this effectively, he warns, "or you could end up connecting a server by mistake to the demilitarised zone (DMZ)."
An even more fundamental piece of advice is to use appropriately sized switching products. "I've seen people pick a great server and then put a cheap switch in the middle," warns Black. "The minute you launch the third virtual machine, the thing just tanks." Port-level buffer sizes are a critical factor, he adds.
Many switches now support virtualisation, enabling them to function as multiple virtual network interface cards for virtual machines. Neterion, for example, sells switches that present themselves in a VMware-enabled server as multiple physical NICs.
In short, it makes sense to virtualise aspects of your network when virtualising the resources connected to it. It makes your infrastructure more agile, and able to meet the performance and capacity demands that can arise in a virtualised environment. But this approach requires a level of discipline and sophistication that some network managers may have missed. Be sure that your virtualisation effort involves the requisite level of planning and pre-specification to avoid tears later on.
This was first published in November 2008