Lenovo’s Secret HPC Weapon: Neptune’s Trident

In the struggle for data center ascendance, the forces of electricity and heat do battle with forces of density and computational power with each step up the Moore’s Law ladder. As more cores are added to each successive CPU release, computational power increases. However, the electricity needed also increases, which produces more heat. Taming these forces and delivering superior performance while consuming less electricity is truly a task for the gods. Lenovo has enlisted the help of Neptune, the god of the sea, to accomplish that feat.

Neptune’s (for those who remember their Roman gods) weapon of choice was his trident, a three-pronged spear he used to tame the sea. This was the inspiration behind Lenovo’s liquid cooling “trident”—Direct to Node (DTN), rear door heat exchanger (RDHX), and hybrid, which combines both air and liquid cooling—that comprise Lenovo’s Neptune* technologies. Lenovo systems with Neptune technologies will deliver superior performance within a smaller energy envelope, using electricity more efficiently, to reduce ongoing OPEX.

Power Usage Efficiency (PUE) is a key measure of how much power in the data center is used for non-computing tasks, determined by dividing the total facility power by the total IT power, with an ideal score being 1.0. Perhaps in the future, Neptune technologies will make PUE as important a specification as gigahertz or teraflops when choosing a server platform.

Neptune: Delivering Performance Improvements through New Approaches to Cooling

Rear door heat exchangers have become the entry step in liquid cooling. Essentially, an RDHX is a big radiator, just like that found in a car, attached to the rear door of the rack. The RDHX absorbs the heat discharged from air-cooled systems. This process diminishes the hot air expelled into the “hot aisle”, reducing overall air conditioning costs and allowing slightly higher system performance. The PUE for a RDHX is around 1.41 using chilled water. While RDHX solutions can continue to utilize standard, air-cooled compute, storage and networking components, the addition of water cooling via RDHX can further improve PUE.

Earlier this year, the ThinkSystem* SD650 with DTN liquid cooling began shipping to customers around the world. Lenovo’s thermal engineers have been developing liquid-cooled systems since 2012. The new ThinkSystem SD650 utilizes warm (50°C) water instead of air to cool the components, including the CPUs and memory, as well as the PCI Express* and voltage regulators. ThinkSystem SD650 servers have no system fans, do not require water chillers, and operate at lower temperatures compared to standard air-cooled systems. Because water conducts heat more efficiently, customers can run Intel® Xeon® Scalable processors up to 240W+, where traditional air-cooled systems would be limited to 165W, thus delivering more computational horsepower, (Neptune was also the god of horses!). The result—greater performance with 30-40% lower data center power consumption and a PUE rating of 1.12.

In the future, Neptune technologies will expand to different combinations and components. Our engineers are working on the third prong of the trident: Hybrid cooling, the combination of air and liquid in the same chassis to optimize either performance or energy consumption depending on pre-set policies. We’re also working to improve and refine our Energy Aware Runtime (EAR) software, which can dynamically throttle the frequency of the CPU and memory to optimal levels for a given workload. EAR can also monitor and control elements from a chassis level, down to the fan speed of an individual node.

So, who would benefit from Neptune technologies? Data centers where the monthly electricity bill is greeted with “How much?!!” for starters. Similarly, those whose data center infrastructure was not built to support the increasing heat profile of air-cooled systems. Lastly, those who are trying to drive as much computational performance as is possible. Systems with processors that consume 200W and higher are forcing HPC cluster deployments to strongly consider Neptune-enabled liquid cooling to keep moving up the ladder of improved generation-to-generation compute performance promised by Moore’s Law.

Neptune is Leading the Voyage to Exascale

These are exciting times for the HPC community, as we continually push beyond today’s performance limitations. To get maximum performance, we are beginning to see HPC server power consumption increase by 2x from just a few years ago3. Moreover, exascale computing is on the horizon. Even with all of exascale’s promise, there are hurdles that need to be overcome before we can instantly map DNA or forecast weather down to the house level.

As we continue the voyage toward ExaFLOP systems, our investments in the alternative cooling technologies within Neptune will enable these massive scale systems to overcome some of these hurdles. Along the way, the benefits of this effort should cascade to all data centers. So one day soon, if you see a large merman with a trident hanging out with your system administrators, don’t worry. He’s just there to run your systems faster and save you money.


Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors. Performance tests, such as SYSmark and MobileMark, are measured using specific computer systems, components, software, operations and functions. Any change to any of those factors may cause the results to vary. You should consult other information and performance tests to assist you in fully evaluating your contemplated purchases, including the performance of that product when combined with other products. § For more information go to www.intel.com/benchmarks.Intel technologies’ features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. No computer system can be absolutely secure. Check with your system manufacturer or retailer or learn more at www.intel.com.

1 https://mediatum.ub.tum.de/doc/1399734/1399734.pdf
2 https://mediatum.ub.tum.de/doc/1399734/1399734.pdf
3 Based on the power consumption of systems such as Eli Lilly and Company BladeCenter HS21 (~20kW power consumption in 2006), LRZ - SuperMUC-NG Lenovo SD650 (~46 kW power consumption in 2018)

Published on Categories High Performance Computing, UncategorizedTags , , , ,
Vinod Kamath

About Vinod Kamath

As a cooling architect for Lenovo’s ThinkSystems, keeping servers from overheating has been Vinod’s primary role as a Lenovo Difference Maker. For more than 25 years, Vinod has been one of the world’s preeminent experts on data center cooling, driving innovations that have contributed to dramatic improvements in high performance computing (HPC) while maximizing energy efficiency. After beginning his career at IBM as a cooling architect for System X, Vinod has been a key contributor on projects that have changed the dynamics for energy efficient computing, developing innovative new cooling architectures for servers. Among Vinod’s chief innovations as a Lenovo Difference Maker is spearheading the development of Lenovo’s warm water-based cooling systems and was part of the team that delivered the first high-scale warm water cooling system for servers. Vinod is passionate about developing green computing solutions and is actively involved with the Energy Efficient High Performance Computing Workgroup, which is a primary driver of energy efficient computing solutions whose efforts are helping efforts to get an exaflop HPC solution on the horizon in a timely manner. Vinod’s interest in solving the difficult problems of efficient cooling has led to generation of over 50 issued patents in the areas of thermal engineering and systems packaging. He has also authored several papers in fluid mechanics and electronics cooling and will be publishing this first book in the arena or Energy Efficient Data Centers at the end of 2018. Vinod has a PhD in mechanical engineering from John’s Hopkins University. In his spare time, Vinod enjoys reading and keeping abreast about the new developments in particle Physics and Cosmology, evolutionary biology, the economics and the theory of finance and business.