Intel Omni-Path Architecture Momentum: What a Difference a Year Makes

Intel’s high-performance computing (HPC) fabric, Intel® Omni-Path Architecture (Intel® OPA), continues to make strides, driven by its performance and price/performance. Organizations choose Intel OPA because it offers 100 Gbps with very high message injection rates and low latency, enabling the “Top500”-level performance seen in today’s leading institutions, such as Cineca, Joint Center for Advanced HPC, and Texas Advanced Computing Center. Additionally, Intel OPA’s 48-port radix switch silicon enables simpler networks that deliver full bi-sectional bandwidth with less hardware and lower networking costs, allowing system architects to spend more of their budget on compute power.

For the HPC interconnect market segment, a lot happened in just a year -- especially when it comes to Intel OPA. Take a look at the June 2017 list for the Top500.[1] Intel OPA is the only interconnect to grow its share over the last year. All other interconnects either stayed flat or lost share: Intel OPA added 30 new listings, while InfiniBand* lost 19 (down 10%) and Ethernet lost 11 (down 5%) over the year[2]. The Top500 is a good proxy for the HPC marketplace, and uptake within the Top500 list is indicative of the growing attractiveness of Intel OPA for today’s supercomputer builds. Digging deeper into the Top500 leads to even more interesting findings.

Graph - Change in # Systems from June'16 to June'17
Interconnect Change (# of Systems) Year-to-Year
  • Growth - More system builders are choosing Intel OPA. There are 36% more systems with Intel OPA in the June 2017 list than in the November 2016 list six months ago[3].
  • Performance - Of the 100 Gbps interconnects on the Top500, Intel OPA-based machines deliver 73% more FLOPS than InfiniBand EDR: 67.1 petaFLOPS for Intel OPA compared to 38.7 petaFLOPS for EDR[4]. Intel OPA-based clusters claimed 7.6 percent of Top500 Rmax performance[5].

    Intel® Xeon® Processor Efficiency
  • Performance Efficiency – The Intel® Xeon® processor runs more efficiently on the Intel fabric: 81.9% efficiency on Intel OPA vs. 74.3% efficiency on EDR[6].
  • Scale & Stability - Intel OPA is the only 100Gbps fabric used in the top 15 listings[7], with more than one-quarter of these systems now using Intel OPA – not bad for an interconnect technology just a little over a year old.
  • Power Efficiency - Intel OPA was used in the number one Green500 machine[8]. All 38 systems in the Top500 that were included in the Green500 use Intel OPA[9].

Intel’s Latest Processor Launches in the Top500

Intel also just launched its most advanced Intel Xeon processor, the Intel Xeon Scalable processor. Two of the Top500 supercomputers chose Intel’s new CPU for their machines. MareNostrum at Barcelona Supercomputing Center landed at number 13 and QURIOSITY at BASF placed 65 on the Top500. Both systems use Intel OPA.

Examine for yourself how Intel OPA has placed in the Top500 by visiting www.top500.org. For more information about Intel OPA, please visit http://www.intel.com/content/www/us/en/high-performance-computing-fabrics/omni-path-architecture-fabric-overview.html.

[1] https://www.top500.org/lists/2017/06/; https://www.top500.org/lists/2016/11/; https://www.top500.org/lists/2016/06/; https://www.top500.org/lists/2003/06/; https://www.top500.org/lists/2003/11/; https://www.top500.org/lists/2004/06/.[2] Based on the June 2017 Top500 List and June 2016 Top500 List. https://www.top500.org/list/2017/06/, https://www.top500.org/lists/2016/06/
[3] Based on number of June 2017 Top500 List Intel OPA systems and number of November 2016 Top500 List Intel OPA systems. https://www.top500.org/list/2017/06/, https://www.top500.org/lists/2016/11/
[4] Based on FLOPS for June 2017 Top500 List Intel OPA systems and FLOPS for June 2017 Top500 List InfiniBand EDR systems. https://www.top500.org/list/2017/06/
[5] Based on FLOPS for June 2017 Top500 List from Intel OPA systems and total overall FLOPS for all June 2017 Top500 List systems. https://www.top500.org/list/2017/06/
[6] Based on average percent efficiency for June 2017 Top500 List Intel OPA systems and average percent efficiency for June 2017 Top500 List InfiniBand EDR systems. Percent efficiency is calculated by dividing Linpack Performance (Rmax) by Theoretical Peak Performance (Rpeak). https://www.top500.org/list/2017/06/
[7] Based on June 2017 Top500 List, four of top 15 listings use Intel OPA. https://www.top500.org/list/2017/06/
[8] Based on June 2017 Green500 List. https://www.top500.org/green500/lists/2017/06/
[9] Based on June 2017 Green500 List and June 2017 Top500 List. https://www.top500.org/green500/lists/2017/06/, https://www.top500.org/list/2017/06/