Data Center Design Is the Key to Industry-Leading PUEs

Data Center Design - By the Numbers

When you think about the connection between Intel and the growing data center market, the first thing that may occur to you is that Intel is a major supplier of data center infrastructure – server processors, network components, and so on. But as Intel IT’s Chief Technology Officer and senior principal engineer, I can also tell you that Intel’s IT department designs, builds, and operates world-class data centers that provide the computing power necessary for Intel’s global chip design efforts.

You might ask why Intel chooses to invest in new corporate data centers in certain situations, rather than simply buying more cloud-based computing power. The answer lies in efficiency and optimization. Intel’s corporate data centers support four major areas of activity: design computing, office and general-purpose computing, manufacturing computing, and enterprise applications that support eBiz. Here, I’m going to focus on design computing.

Chip Design Drives Data Center Need

In the last decade and more, Intel chip design computing has grown 4,600% (46x) to meet ever-growing Intel silicon design complexities. This growth in complexity has resulted from the following trends:

  • Ever-decreasing chip size – from 90nm in 2004 to current 10nm designs
  • The integration of an ever-increasing number of CPU instructions, features, and capabilities that require more and more devices and transistors
  • Ceaseless pressure for faster time to market, requiring constantly accelerating design cycles

Intel Designs Two Extremely Energy-Efficient, High-Density Data Centers

Intel IT: Extremely Energy-Efficient, High-Density Data Center Design White PaperIn response to this growth in chip design complexity, and as part of our effort to strategically transform data centers to achieve significant business results, Intel IT used design best practices to convert two vacant silicon-wafer-fabrication building modules into extremely energy-efficient, high-density, 5+ MW data centers. You can read about the details of these two data center designs in our recent white paper, “Intel IT: Extremely Energy-Efficient, High-Density Data Centers.”

In a nutshell, each of these data centers has its own unique design and cooling technologies that are built at extremely low $/kW. They are both designed to be environmentally sustainable through creative use of unused building space and prevailing site environmental conditions.

The first of these two data centers (we’ll call it data center module A) uses outside free-air cooling to support a 5-MW IT load power within 5,000 square feet. Over two years of operation, it has delivered 1.06 PUE (which is 13.33 times more energy efficient than the industry average of 1.8 PUE – using just 6% versus the typical 80% overhead on top of IT useful load). The latest data center module (we’ll call it module B) uses close-coupled evaporative cooling to support the first phase of 5.5-MW IT load power in production with this cooling technique. This module is designed to operate at a 1.07 annual PUE and will have a total power capacity of all of Intel’s existing high-density and legacy data centers combined once we complete the remaining phases of construction.

Data Center Design Results

These innovative data center designs are achieving the following results:

  • The rack power density (up to 43 kW per rack) is nearly 1.5 times greater than what we have delivered in the past for high-density computing.
  • The 1,100 W/square-foot cooling density and 1,300 W/square-foot electrical density are 10 times the industry average.
  • Module A used only free-air cooling except for less than 39 hours in 2014 and runs our servers at an air intake temperature of up to 91⁰F. We are saving $1.9 million per year in utility bills and an estimated 44 million gallons of water annually thanks to this design.
  • Module B uses only evaporative cooling-tower water to condition the data center space at up to 91°F supply air and densities greater than 1,000 W/square foot

We are able to achieve the high compute densities demonstrated in these data center design by using more than 60,000 Intel® Xeon® processor-based servers, which offer 51% higher performance per core than previous models. The higher cooling and electrical densities will enable us to continue to support the large growth in compute demand associated with electronic design automation tools, while delivering high performance for application needs.

Annual Performance Report
2016-2017 Intel IT Annual Performance Report

I encourage you to read about the design of these two data centers. Our recently published 2016-2017 Intel IT Annual Performance Report Powering Digital Business Transformation contains more information about new, breakthrough data center technologies that we are using to improve performance and cost efficiency.

Published on Categories Data CenterTags , , , , , ,
Shesha Krishnapura

About Shesha Krishnapura

Shesha Krishnapura is an Intel Fellow and chief technology officer in the Information Technology organization at Intel Corporation. He is responsible for advancing Intel data centers for energy and rack space efficiency, high-performance computing (HPC) for electronic design automation (EDA), and optimized platforms for enterprise computing. He is also responsible for fostering unified technical governance across IT, leading consolidated IT strategic research and pathfinding efforts, and advancing the talent pool within the IT technical community to help shape the future of Intel. Shesha has led the introduction and optimization of Intel® architecture compute platforms in the EDA industry since 2001. He and his team have delivered five generations of HPC clusters and four supercomputers for Intel silicon design and device physics computation. Earlier in his Intel career, as director of software in the Intel Communications Group, he delivered the driver and protocol software stack for Intel’s Ethernet switch products. As an engineering manager in the Intel® Itanium® processor validation group, he led the development of commercial validation content that produced standardized workload and e-commerce scenarios for successful product launches. He joined Intel in 1991 and spent the early years of his Intel career with the Design Technology group. A three-time recipient of the Intel Achievement Award, Shesha was appointed an Intel Fellow in 2016. His external honors include an InformationWeek Elite 100 award, an InfoWorld Green 15 award and recognition by the U.S. Department of Energy for industry leadership in energy efficiency. He has been granted several patents and has published more than 75 technical articles. Shesha holds a bachelor’s degree in electronics and communications engineering from University Visvesvaraya College of Engineering in Bangalore, India, and a master’s degree in computer science from Oregon State University. He is the founding chair of the EDA computing board of advisers that influences computer platform standards among EDA application vendors. He has also represented Intel as a voting member of the Open Compute Project incubation committee since its inception.