Next Generation OCP Twin Lakes Built on Intel Xeon D Processors

By John Bryan, Platform Architecture, Intel and Damien Chong, Platform Architect, Facebook

Proliferation of server makes and models over a span of years in the data center is not a huge deal for most enterprises. They cope with the diversity because they support a diversity of application and can kind of keep things isolated. Moreover, while IT may be integral to their product or service, it is usually not the actual product or service that they sell.

Not so for us.  When we started the Open Compute Project six years ago we launched an industry-wide initiative to share specifications and best practices for creating the most energy efficient and economical data centers. Today we are announcing the Twin Lakes 1S server.

The Twin Lakes 1S server incorporates the energy-efficient capabilities of the Intel Xeon® D-2100 Processor SoC to provide strong compute performance as well as high I/O density.
Figure 1 Twin Lakes 1S Server with the Yosemite V2 System

The Twin Lakes 1S server incorporates the energy-efficient capabilities of the Intel® Xeon® D-2100 Processor SoC to provide strong compute performance as well as high I/O density. The Twin Lakes server is engineered for both single- or multi-node system configurations.

Compute Performance

Twin Lakes compute performance is fueled by an up to 18 core Intel® Xeon® D Processor. Each core supports:

  • Virtual address space of 48 bits and a physical address space of 46 bits.
  • Intel Hyper-Threading Technology (Intel® HT Technology) when enabled allows each core to support two threads.
  • First Level Cache (FLC) 64 KB total. The FLC is comprised of a 32 KB ICU (Instruction Cache) and 32 KB DCU (Data Cache)
  • 1 MB Mid Level Cache (MLC) per core (non-inclusive with the LLC).
  • Intel® Advanced Vector Extensions 512 (Intel® AVX-512) with a single AVX512 fused multiply-add (FMA) execution units Twin Lakes supports the latest generation of Xeon® D Processor SoC, offering significant enhancements over the previous generation. The below table illustrates some of these improvements.
Intel® Xeon® D-2100 Processor Intel® Xeon® D-1500 Processor
CPU Cores Up to 18 with Intel® HT Up to 16 with Intel® HT
Cache LLC: 1.375 MB/Core

MLC: 1 MB/Core

LLC: 1.5 MB/Core

MLC: 256 KB/Core

Memory 4 Channels,

DDR4 1866/2133/2400/2667*

2 Channels

DDR4/DDR3L 1600/1866/2133

PCIe CPU: x32 PCIe Gen 3

  • Twin Lakes uses all 32 lanes

FlexIO: x20 PCIe Gen 3

  • Twin Lakes uses 9 FlexIO PCIe lanes
CPU: x24 PCIe Gen 3 lanes

FlexIO: x8 PCIe Gen 2 lanes

Acceleration Engines Intel® AVX-512

Intel® QuickAssist Technology*

  • Up to 100 Gbps Crypto/Compression
  • 100KOps PKE 2K
AVX256

No built-in Intel® QAT

* Intel® Xeon® D-2100 Processor feature not validated on the Twin Lakes 1S Server

I/O Interfaces

Twin Lakes I/O is routed off-board using standard high-volume edge connector technology, and includes:

  • 28 PCIe Gen3 lanes for general-purpose use
  • 1 PCIe Gen3 lane as a BMC interface
  • 1 10GBase-KR
  • 1 SATA port
  • 1 USB 2.0 port
  • 1 Universal Asynchronous Receiver/Transmitter (UART)
  • 1 I2C bus for server management

In addition, Twin Lakes has two on-board M.2 connectors, which can support SSDs in the 2280 or 22110 form factors. Both M.2 connectors are linked to the SoC CPU through a x4 PCIe Gen3 NVMe channel. An additional M.2 connector can support a 2280 form factor boot SSD, configurable as either SATA or PCIe Gen3 NVMe. The Twin Lakes architecture is illustrated in the following block diagram:

Twin Lakes has two on-board M.2 connectors, which can support SSDs in the 2280 or 22110 form factors.
Figure 2: Twin Lakes Block Diagram

Memory Subsystem

The Intel® Xeon® D-2100 Processor on Twin Lakes supports four channels of DDR4 RDIMMs. The Twin Lakes memory subsystem implementation includes two DIMMs per channel for a total of eight DIMM sockets.  The Intel® Xeon® D-2100 Processor supporting RDIMM capacities up to 128 GB at one DIMM/channel or 64 GB at two DIMMs/channel, Twin Lakes can support half a Terabyte of memory.

The Twin Lakes memory subsystem implementation includes two DIMMs per channel for a total of eight DIMM sockets.
Figure 3: Twin Lakes Illustration Showing Memory Subsystem

Network Interface

The Twin Lakes 1S server can use either an external network interface controller on the platform through its PCIe interface, or the Intel® Xeon® D-2100 Processor’s integrated 10GbE network controller. When the Intel® Xeon® D-2100 Processor’s integrated network controller is used, SMBus or NC-SI can be used for sideband communication. The integrated 10GbE controller supports routable and scalable RDMA, ideal for large segmented networks in both private and public clouds. The controller supports network virtualization overlays, to allow abstraction of the network for cloud flexibility, and Intel® Ethernet Flow Director for advanced traffic steering.

Server Management

The Twin Lakes 1S server uses a Bridge IC (Texas Instrument’s Tiva microcontroller) as the management controller. This Bridge IC also serves as a bridge between the system BMC and the Intel® Xeon® D-2100 Processor. The Bridge IC manages the 1S server on behalf of the BMC on the platform and bridges the BMC and Intel® Xeon® D-2100 Processor’s internal management controller, the Intel® Manageability Engine. A dedicated point-to-point I2C bus is used to maximize the communication bandwidth between the BMC and the Bridge IC.

About the Authors: 

 

 

 

 

Damien Chong, Platform Architect, Facebook

Damien Chong is Hardware Engineer from the Compute Server Design Team at Facebook. He is the overall hardware owner of Facebook’s OCP 1-Socket Platform, Yosemite V2. Before joining Facebook, he served as Senior Hardware Engineer and Platform Power Delivery Architect in National Instruments and Intel Corporation respectively. He holds 1 US patent. He received his First Class Honours B.Eng in Electrical and Electronics Engineering from Nottingham Trent University, United Kingdom.

 

 

 

 

John Bryan, Platform Architecture, Intel

John Bryan is a Platform Architect at Intel Corporation, working on high-density servers solutions for the data center. John has collaborated with Facebook on the Mono Lake and Twin Lakes one-socket servers based on Intel Xeon-D SOCs. John has designed servers and supercomputers at Intel for 25 years and holds BS and MS degrees from Stanford University.