Need to Get the Most out of 10 Gigabit? Use Nehalem.

Sometimes the next step up is a big one. The Intel® Xeon® processor 5500 series (formerly codenamed “Nehalem”) is one of those kinds of steps.

Over the last few years 10 Gigabit has started to take off, but there have always been some negative mutterings: “Why do I need 10 Gigabit?”, “Why do we need this much bandwidth?” or “My server can’t support 10 Gigabit per second bidirectional traffic anyway.” Despite the volume of 10 Gigabit products shipped, there is still the reality that if you intend to use the entire 20 Gbps (10G both directions) or heaven forbid you try to use 40 Gbps with a dual port product; you will likely be sorely disappointed with the results.

The reason for this is simple. Most current mainstream servers and 10 Gigabit products don’t support the intense usage models needed to drive that much network I/O and they also don’t have the memory architecture to unleash the full potential of dual 10 Gigabit links.

Luckily, that all just changed with Intel® Xeon® processor 5500 series.

In addition to the great processing improvements that the Intel® Xeon® processor 5500 series brings to the table, Intel has also introduced our third generation 10 Gigabit product, the Intel® 82599 10 Gigabit Ethernet Controller which provides two ports, and new capabilities and enhancement to the 10 Gigabit product landscape that help unshackle the new processor from its predecessor’s network I/O handcuffs and unleashes blazing performance in a variety of usage models. These improvements, coupled with the new architecture of the Xeon 5500 provide a symbiotic processor-networking combination that makes new usages possible and expands server and datacenter computing by a big leap… not just a baby step.

One of the key changes with Intel® Xeon® processor 5500 series architecture is a step function improvement in the internal system I/O. The new local memory controller design, faster cache architecture, and support for DDR3 help push Xeon 5500 to be able to support peak memory bandwidth of ~32 Gigabytes, per socket. In a dual socket system this provides for ~64 Gigabytes of bandwidth which is dramatically more than the previous generation server configuration. In addition, the new Intel® QuickPath Interconnect (Intel® QPI) improves the speed both for inter-Processor communication as well as a faster path to the I/O hub. Finally, PCI Express* 2.0 I/O Bus support has been added to improve the entire data path from Processor to the 10 Gigabit Ethernet link.

Taken together, the above improvements are a performance game changer for 10 Gigabit Ethernet.

The chart below** shows the previous generation Intel® Xeon® paired with the previous generation Intel 10 Gigabit Ethernet Controller compared to the latest platform using the newest Intel silicon for both processor and networking. Not only is the performance better in 1-4 port configurations, but the performance scales dramatically better to above 50 Gigabits per second of total LAN throughput in a four port configuration vs. *just* 17 Gigabits on the previous generation! A complete platform architecture solution makes this huge improvement possible.

82599 + Xeon 5500.jpg

Now, it’s great that Intel® Xeon® processor 5500 series coupled with the Intel® 82599 10 Gigabit Ethernet can deliver such raw performance, but there is the forever nagging question of usage model. Luckily, the new headroom breathes new life into both Virtualization and storage over Ethernet usages (both of which I’ve talked about here and here) and provides new opportunities to more efficiently utilize your network link.

Intel® Xeon® processor 5500 series allows the vision of consolidation in the datacenter to scale new heights, increasing the number of Virtual Machines (VM) that can effectively live inside a single system enclosure. Each incremental VM will add additional network I/O that is already starting to exhaust a 4 or 8 port single gigabit interface configuration with today’s server capabilities. As more VMs are added in the Xeon 5500 generation, 10 Gigabit will no longer be seen as optional; it will be required. For its part, the Intel® 82599 10 Gigabit Ethernet Controller supports Intel® Virtualization Technology for Connectivity (Intel® VT-c) to improve overall system performance in virtualized server environments. Intel VT- c includes hardware optimizations that help reduce I/O bottlenecks, boost throughput and reduce latency. Components of Intel VT-c include VMDq, and VMDc. VMDc consists of SR-IOV which I’ve covered before, and the ability to support VM mobility; a critical usage model for modern a IT deployment. All together, server systems can support more VMs, more throughput, more flexibility and better performance in a datacenter environment.

Finally, the additional capabilities of the Intel® 82599 10 Gigabit Ethernet Controller product surrounding support for FCoE offloads and full support for the new Data Center Bridging (DCB) standards provide an opportunity for storage convergence over Ethernet in either a datacenter using a Fiber Channel SAN environment or an IT environment more focused on iSCSI. On the performance side of things, iSCSI acceleration along with FCoE data path offloads are supported in the Ethernet controller, and on the processor there is support for the CRC instruction set which insures iSCSI data integrity while minimizing processor overhead.

The ability to converge at least part of the additional storage infrastructure onto Ethernet is just another factor driving massively increased data rates over Ethernet… luckily, the Intel® Xeon® processor 5500 series and the Intel® 82599 10 Gigabit Ethernet Controller solutions are up to the task.

Over the past few days, there has been a lot of noise around Intel® Xeon® processor 5500 series and the many other platform components that help it shine its brightest. Improved processing power, memory controller bandwidth, faster and redesigned FSB, and improved 10 Gigabit networking all converge together to provide a fantastic performance, convergence, scalability and power story. Intel’s strong history in the server and processor markets, coupled with over 25 years in Ethernet makes this latest release a natural evolution of technology. Together these capabilities, along with the improved 10 Gigabit features and performance, are helping to transform the datacenter. It will be denser, more power efficient, more performant, and more consolidated with capabilities like FCoE and iSCSI.

As for “Why do I need 10 Gigabit?” We have the answer, and it’s the new Xeon®.

Ben Hacker


** Source. Intel. Mar 2009. Up to 2.5x performance compared to Intel® Xeon® processor 5300 series. Performance result of a bandwidth intensive network benchmark (IxChariot). Network throughput was measured on 64KB I/O size transfers between the test system and multiple network targets. Intel pre-production system with two Quad-Core Intel® Xeon® processor 5500 series CPUs (2.93 GHz), 12 GB memory (6 x2GB DDR3 - 1066MHz) vs. Intel Production system with two Intel® Xeon® processors X5365 (3.0GHz, 1333MHz FSB), 8 GB memory (8 x 1 GB DDR 2 - 667). Windows Server 2008, stock unmodified installation.