At Intel Imaginative and prescient 2024, Intel had lots to say about AI and what it’s been engaged on in that space. The corporate introduced a brand new AI accelerator known as Gaudi 3, its plans to collaborate on an open platform for enterprise AI, and subsequent technology processors.
Gaudi 3 makes use of Ethernet to attach tens of 1000’s of accelerators, which the corporate believes will allow a “important leap in AI coaching and inference for international enterprises trying to deploy GenAI at scale.”
Every accelerator can run 64,000 operations in parallel, which helps the computational complexity required by deep studying algorithms. Its reminiscence capability is 128 GB and it additionally has 3.7 TB of reminiscence bandwidth and 96 MB of obtainable on-board static RAM. In accordance with Intel, these reminiscence specs make it potential to effectively serve LLMs and multimodal fashions.
The software program Gaudi runs on integrates with PyTorch and gives optimized fashions from Hugging Face, which the corporate says makes it straightforward to port fashions throughout completely different {hardware} varieties.
Gaudi 3 additionally introduces a peripheral element interconnect categorical (PCIe) card that’s useful for workloads like fine-tuning, inference, and retrieval augmented technology.
In comparison with its competitor Nvidia H100, Intel expects Gaudi 3 to be 50% sooner to coach throughout Llama2 with 7B and 13B parameters and GPT-3 with 175B parameters. It additionally is anticipated to have 50% extra throughput usually and 40% extra for inference power-efficiency, in comparison with Nvidia’s.
Intel anticipates making Gaudi 3 obtainable to producers, together with Dell Applied sciences, HPE, Lenovo, and Supermicro, within the second quarter of this yr.
“Within the ever-evolving panorama of the AI market, a big hole persists within the present choices,” stated Justin Hotard, govt vp and basic supervisor of the Information Heart and AI Group at Intel. “Suggestions from our prospects and the broader market underscores a need for elevated selection. Enterprises weigh issues similar to availability, scalability, efficiency, price, and power effectivity. Intel Gaudi 3 stands out because the GenAI different presenting a compelling mixture of value efficiency, system scalability, and time-to-value benefit.”
Alongside the announcement of Gaudi 3, the corporate additionally introduced that it was collaborating with various firms to create an open platform for AI within the enterprise.
To help this effort, Intel will probably be releasing reference implementations for GenAI pipelines of Intel Xeon and Gaudi-based techniques, publish a technical conceptual framework, and add extra infrastructure capability within the Intel Tiber Developer Cloud.
The opposite firms who’re working collectively on this challenge embody Anyscale, Articul8, DataStax, Domino, Hugging Face, KX Methods, MariaDB, MinIO, Qdrant, RedHat, Redis, SAP, VMware, Yellowbrick, and Zilliz.
And at last, the corporate introduced the subsequent technology of its Intel Xeon processors. The brand new Intel Xeon 6 processors embody Environment friendly-cores (E-cores) and Efficiency-core (P-cores). The E-cores supply a 4x efficiency enchancment and a pair of.7x higher rack density than the 2nd technology Intel Xeon processors. P-cores add help for the MXFP4 information format, lowering token latency by 6.5x in comparison with the 4th technology Intel Xeon processors.
In accordance with Intel, the Xeon 6 processors with E-cores will launch this quarter and processors with P-cores will launch after that.
The corporate additionally teased that the subsequent technology of Intel Extremely processors will launch later this yr and may have over 100 platform tera operations per second (TOPS) and over 45 neural processing unit TOPS.
“Innovation is advancing at an unprecedented tempo, all enabled by silicon – and each firm is shortly turning into an AI firm,” stated Pat Gelsinger, CEO of Intel. “Intel is bringing AI in all places throughout the enterprise, from the PC to the info heart to the sting. Our newest Gaudi, Xeon and Core Extremely platforms are delivering a cohesive set of versatile options tailor-made to fulfill the altering wants of our prospects and companions and capitalize on the immense alternatives forward.”
