Right off the bat some may balk at the thought that High Performance Computing (HPC) has a place in the enterprise. While that may have been the case in the past where it was relegated to academic and research organizations, times have changed. Today, HPC does have a growing place in the enterprise and this post is intended to outline why and how.
Understanding what HPC is and the drivers for change
Essentially, HPC is an incredibly powerful computing infrastructure built specifically to conduct intensive computational analysis. Examples include physics experiments that identify and predict black holes. Or modeling genetic sequencing patterns against disease and patient profiles. In the past year, the Amaro Lab at UC San Diego performed modeling on the COVID-19 coronavirus to an atomic level using one of the top supercomputers in the world at the Texas Advanced Computing Center (TACC). I hosted a webinar with folks from UCSD, TACC and Intel discussing their work here.
Those types of compute intensive workloads are still happening. However, enterprises are also increasing their demand for compute intensive workloads. Enterprises are processing increasing amounts of data to better understand customers and business operations. At the same time, edge computing is creating an explosive number of new data sources.
Due to the sheer amount of data, enterprises are leveraging automation through the form of machine learning and artificial intelligence to parse the data and gain insights while making faster and more accurate business decisions. Traditional systems architectures are simply not able to keep up with the data tsunami.
HPC in the enterprise
This is where HPC enters the enterprise. While traditional HPC companies like Cray may not be household names within the enterprise, companies like HPE are. HPE’s acquisition of Cray and furthermore integrating it into their Greenlake portfolio bridge the gap for enterprises looking to expand their compute capabilities into HPC. This is just one example of companies making the move.
Over time, HPC architectures are integrating with traditional enterprise architectures to smooth the process of systems integration between the two architectures. In many ways, HPC becomes an extension of the compute capabilities for enterprises.
HPC is not replacing traditional enterprise architectures. It is, however, augmenting in ways that enterprises need as they increase their demand for data processing at scale.
Leveraging HPC: on-premises and in the cloud
There are many ways organizations can leverage HPC environments today. One of those ways is in the cloud using specific instances geared toward intensive workloads and large data sets. The other way is with on-premises systems designed to provide compute intensive workloads with large data sets.
Large enterprises are already working with HPC in their environments both on-premises and in the cloud. There are advantages to both approaches, but often the decision is based on factors including maturity, architecture, data sets, access to cloud, regulatory, compliance and privacy.
Enterprises will need to determine which of the approaches best fits their needs. The one thing that is clear is that HPC is destined for broad enterprise use in the short to near term. As companies look to leverage data in a more meaningful way, the sheer scale of the data coming at them will require a different way of thinking.
