As a long-time supporter of high performance computing (HPC), I am excited to see more businesses and industries gaining access to HPC’s transformative power to deliver their next groundbreaking product or world-changing scientific discovery. It also is invigorating to see the new and different ways that HPC capabilities are being delivered and enabled by this community.
HPC is evolving to become more data-centric. Traditional HPC focused on Simulation and Modeling workloads, but this new era of data-centric HPC (DC-HPC) adds High Performance Data Analytics (HPDA) and Artificial Intelligence (AI). Increasing data opportunities are driving ever-greater demand for HPC capabilities, and DC-HPC is the infrastructure that future data workflows will require: big compute, big memory, and low latency all are critical.
HPC also is expanding into new users and new usages. This means that how people are using HPC resources is changing. While traditional on-prem HPC centers will remain critical to expanding research and discovery, we are seeing a rise in HPC in the cloud for public cloud, hybrid, and even on-prem HPC systems.
Demand for HPC from broader audiences and demand for greater HPC resources for converged workflows is driving significant growth in the overall market for HPC. Forecasts indicate that the growth of HPC over the next five years will exceed the growth we saw in the previous five and that the total HPC market could reach $44 billion by 2022.1
At Intel, we support HPC-based innovation in all its forms, from exascale supercomputers to our Intel® Select Solutions for High Performance Computing – which deliver complete and validated systems to customers – to new cloud-based delivery methods. With the European ETP4HPC initiative’s recent adoption of a forward-looking blueprint for strategizing HPC research through 2027 and the yearly ISC High Performance conference this week in Frankfurt, Germany, it’s an exciting time for an HPC community that is growing in size, diversity, and the ways it enables discovery.
Why HPC in the Cloud?
Cloud-delivered HPC makes sense for a lot of organizations, and for a lot of different reasons. Key benefits we’ve seen in the market thus far have included:
- HPC in the cloud enables you to manage demand surges. HPC clusters usually have a very high utilization, and demand on these systems is often greater than the systems’ capacity. It can be beneficial to extend on-premises workloads to the cloud to increase user productivity and optimize utilization of software licenses. Additionally, this can enable users to get results more quickly.
- HPC in the cloud facilitates collaborative research and development across teams and organizations. Tethers to cloud services can make it easier for users to share an environment, configure workloads, team up in parallel, or view results together.
- HPC in the cloud can provide access to huge new compute resources. Sometimes even large organizations don’t have the compute resources to solve their most pressing problems in an acceptable timeframe. Cloud resources can give access to millions of cores of compute for only the time period that such resources are required.
- HPC in the cloud gives access to the latest, greatest HPC hardware and software technologies. This makes HPC in the Cloud a great way to evaluate new technologies without a large upfront investment.
With these and other benefits, we expect cloud delivery of HPC to continue to grow in the coming years. Use of cloud-delivered HPC may also drive further growth for on-premises and hybrid HPC systems, as experimental cloud-based HPC projects lead to more regular use of HPC throughout the organization and increased demand for local HPC resources.
Cloud HPC in Action
Cloud providers and HPC customers around the world are showing that Intel technologies provide the same excellence for cloud HPC as they provide for more traditional systems. On the occasion of ISC, we are proud to highlight several European companies enabling cloud HPC services, as well as an example from the United States.
Cambridge-based researchers Satavia have tapped European cloud provider Verne Global’s hpcDIRECT platform to expand their environmental and aeronautical research. hpcDIRECT provides Satavia access to 10,000 bare-metal compute cores from Dell EMC* based on Intel® architecture. Satavia will use hpcDIRECT to, for example, better map high-altitude ice clouds, which can pose a hazard to aircraft.
Leading Nordic high-density computing provider Advania Data Centers now provides a cloud-based offering for enterprise customers using highly-demanding AI and HPC applications for competitive advantage. Advania HPC as a Service streamlines the acquisition of HPC resources, enabling a broader range of users to access the latest HPC technologies, while still receiving a high quality of service.
HPC in the cloud service provider Gompute, based in Sweden and with offices in Spain, Germany, and the United States, recently extended its cloud HPC resources with Intel Xeon Scalable processors and Intel® Omni-Path Architecture (Intel® OPA). Gompute reports that its customers often turn to its cloud services for workloads like computer aided engineering (CAE), crash simulation, structural analysis, and computational fluid dynamics (CFD), and look to Gompute at times when HPC capacity needs to rapidly expand or when organizations lack in-house technical computing expertise.
Texas Advanced Computing Center’s (TACC’s) forthcoming Frontera supercomputer, which is based on Second Generation Intel® Xeon® Scalable processors and includes Intel® Optane™ DC Persistent Memory, will have a cloud component, with Amazon*, Google*, and Microsoft* all participating in the project. This will provide TACC’s users the ability to experiment with the latest architectures that cloud service providers deliver as well as providing TACC new information on considerations for the design and deployment of future HPC systems.
Learn More about HPC in the Cloud
Data processing demands from new users and new usages are driving more and more need for HPC in the cloud to facilitate data-centric HPC applications. As a result, the market for HPC usage in public clouds is predicted to grow from $1.9 billion in 2017 to more than $5.5 billion by 20222—a small fraction of a $40+ billion total market, but a fraction that is growing and will continue to grow. I can’t wait to see what HPC in the cloud enables for organizations and enterprises that previously have not had access to such resources, especially as big data, AI, and HPC continue to converge.
For more information on Intel technologies for HPC, please read Dr. Bill Magro’s blog on HPC cloud resources, stop by Intel Booth #F-930 at ISC, visit intel.com/isc and intel.com/hpc, and follow along on Twitter at @IntelHPC and @Trish_Damkroger.
Intel technologies’ features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. No computer system can be absolutely secure. Check with your system manufacturer or retailer or learn more at intel.com.
Intel, the Intel logo, Xeon, and Optane are trademarks of Intel Corporation or its subsidiaries in the U.S. and/or other countries.
*Other names and brands may be claimed as the property of others.
© Intel Corporation
1 Source: HPC Cloud Market Spending Size and Forecast, 2013 to 2022, Hyperion Research, January 2019.
2 Source: HPC Cloud Market Spending Size and Forecast, 2013 to 2022, Hyperion Research, January 2019.