Nvidia and Intel unveil superior HPC initiatives, bolstering AI capabilities at SC2023

The world’s quickest supercomputers are getting quicker with each Nvidia and Intel racing to speed up essentially the most highly effective computing programs on Earth, with a giant emphasis on AI energy.

On the Supercomputing 2023 (SC23) convention in Denver in the present day, the record of the world’s quickest 500 supercomputers was launched. In a single type or one other, all of the programs have elements from Nvidia or Intel, and in lots of circumstances each. The occasion can also be a showcase to speak concerning the subsequent technology of supercomputers which can be being constructed, what applied sciences they use and the way they are going to be used. 

For Nvidia, the large new system it’s a part of is the JUPITER supercomputer hosted on the Forschungszentrum Jülich facility in Germany. JUPITER may have 24,000 Nvidia GH200 chips and when accomplished, would be the strongest AI supercomputer ever constructed in keeping with Nvidia with over over 90 exaflops of efficiency for AI coaching. Nvidia can also be utilizing the occasion to element a sequence of recent improvements and AI acceleration silicon together with the H200 and a quad configuration for the Grace Hopper GH200 superchip.

To not be outdone, Intel is highlighting its work on the Aurora supercomputer on the Division of Vitality’s Argonne Nationwide Laboratory that’s getting used to construct a 1 Trillion (that’s with a T and never a typo) parameter giant language mannequin (LLM). Intel can also be offering new insights into the following technology of AI acceleration and GPU know-how because it ups the aggressive ante towards rival Nvidia.

Nvidia advances Grace Hopper superchip to construct essentially the most highly effective AI system in historical past

Nvidia first introduced that the Grace Hopper superchip, which mixes CPU and GPU capabilities entered full manufacturing in Might. These chips  have now discovered their method into essentially the most highly effective supercomputers.

“With the introduction of Grace Hopper, a brand new wave of AI supercomputers are rising,” Dion Harris, Director of Accelerated Information Heart Product Options at Nvidia mentioned in a briefing with press and analysts.

The Grace Hopper GH200 powers the JUPITER supercomputer, which Nvidia sees as a brand new class of AI supercomputer. The AI energy of the JUPITER can be used for climate prediction, drug discovery, and industrial engineering use circumstances. JUPITER is being inbuilt collaboration with Nvidia, ParTec, Eviden and SiPearl. 

JUPITER is utilizing a brand new configuration for the GH200 that dramatically delivers extra efficiency. The system makes use of a quad GH200 structure, which because the title implies, makes use of 4 GH200’s in a system node. 

“The quad GH200 options an progressive node structure with 288 Neoverse ARM cores able to attaining 16 Petaflops of AI efficiency with 2.5 terabytes a second of excessive velocity reminiscence,” Harris defined. “The 4 method system is related with excessive velocity NV hyperlink connections to the chip permitting for full coherence throughout the structure.”

In whole the system contains 24,000 GH200 chips which can be related by way of Nvidia’s Quantum-2 InfiniBand networking. The JUPITER isn’t the one system that may use the quad GH200 strategy, in actual fact Nvidia can be utilizing the identical strategy in different supercomputers as nicely.

As a part of the SC23 information, Nvidia can also be saying the standalone H200 silicon. Whereas the GH200 integrates each CPU and GPU, the H200 is simply the discrete GPU. The NVIDIA H200 can be supplied on Nvidia HGX H200 server boards.

“The HGX H200 platform with quicker and extra excessive velocity reminiscence will ship unbelievable efficiency for HPC and AI inference workloads,” Harris mentioned.

Intel GPU efforts proceed to advance supercomputer powers

Intel can also be making a really robust displaying at SC23 with its HPC and AI applied sciences. 

In a briefing with press and analysts, Ogi Brkic, VP and Normal Supervisor, Information Heart and AI/HPC Options Class at Intel, detailed his firm’s efforts for AI and HPC acceleration. 

Brkic highlighted the Intel Information Heart GPU Max sequence and Intel Habana Gaudi 2 accelerator as serving to to paved the way for big supercomputing installations just like the Daybreak Part 1 supercomputer on the College of Cambridge within the UK. The Daybreak system, which is at the moment in part 1, is the quickest AI supercomputer within the UK and contains 512  Intel Xeon CPUs and 1,024 Intel Information Heart GPU Max Sequence GPUs.

Aurora, which is being constructed within the U.S by Intel, HP Enterprise, and the US Division of Vitality can be serving to to develop one of many largest giant language fashions (LLMs) in existence. Brkic mentioned that AuroraGPT is a 1 trillion parameter LLM for science analysis. AuroraGPT is at the moment being educated throughout 64 nodes of Aurora, with the goal being to ultimately scale it to the whole supercomputer which has over 10,000 nodes.

“We’ve labored with Microsoft Deepspeed optimizations to make sure that this 1 trillion parameter LLM is obtainable for everyone to make use of,” Brkic mentioned. “The potential functions for the sort of giant language mannequin are unbelievable, each ingredient of science from biology, chemistry, drug analysis, cosmology and so forth, will be impacted by availability of this generative mannequin.”