The Best HPC Benchmark is your own Application

SC09 was over last month and I was gratified by the energy and enthusiasm of the HPC community.   Equally gratifying was seen the growth of the Intel® Xeon® 5500 processor in the Linpack-benchmark based Top500* list in just over 8 months, 95 entries in the Top500.

Part of my job focusing  on HPC  in Intel’s server  competitive and performance marketing team is providing  our sales force and customer with the best intelligence to make informed decisions.   While it might be easy to make a buying decision on who has highest Linpack score alone, I think most of the users out there don’t buy their systems to run Linpack.   Imagine having made a buying decision on Linpack alone during the transition from Hapertown (Xeon 5400) to Nehalem-EP (Xeon 5500).  Both processors, have relatively the same Linpack scores  (~91.7 for X5570 and ~92.5 for the X5470), yet when you compare their  delivered application performance, the X5500 delivers somewhere between ~2X-3X on variety of HPC applications.  The key difference here is the vast bandwidth improvements in the Nehalem architecture.  Others users might look at memory bandwidth alone (as measured by the STREAM benchmark), yet cores can only consumer so much bandwidth and then you become compute bound again.    That’s why you often hear this word in HPC the word and our key HPC focus here at Intel, balance.  Not only balance of raw speeds and feeds, but also, power and cost/TCO.

That’s why it was gratifying attending a standing room only Birds-of-a-Feather session during SC09, Benchmark Suite Construction for Multicore and Accelerator Architectures (slides),  facilitated by Kevin Skadron from the University of Virginia.  It was interesting to hear how current benchmarks are either meeting or not meeting user needs or how a particular benchmark might or might not mirror the “real” workload of a user.   I am looking forward to the dialogue and discussions out this.

In the coming year, the architecture choices for HPC users are varied from x86, RISC, GPU.   Users should look beyond the “whiz-bang” numbers that all of us marketers use to get your attention, focus on only one benchmark that matters and that is your own application.

*Other names and brands may be claimed as the property of others