Is NVIDIA redesigning Blackwell chips due to overheating issue?
Written by Nicole Cappella Thu 21 Nov 2024
NVIDIA’s highly-anticipated Blackwell artificial intelligence (AI) chips are once again in the spotlight, with reports of chips overheating when linked together in 72-chip data centre racks.
This raises concerns about potential delays in AI-focused data centre projects among customers and investors. The issue, raised by reporters from The Information, is drawn from statements from NVIDIA employees, customers, and suppliers.
The overheating concerns come as NVIDIA races to deliver on its promise of cutting-edge AI hardware, critical for clients such as Meta Platforms, Alphabet’s Google, and Microsoft, who are planning to use the Blackwell GPUs to power next-generation AI data centres.
NVIDIA unveiled Blackwell architecture in March, touting it as a transformative leap in AI hardware. The chips, combining two large silicon components into one, deliver 30 times the performance of their predecessors for tasks like chatbot responses and AI inference.
To maximise performance, NVIDIA introduced the GB200 NVL72 racks, designed to link 72 GPUs, 36 Grace CPUs, and nine NVLink switch trays into a liquid-cooled system capable of achieving 720 petaflops for training and 1.4 exaflops for inference. This high-density design includes two miles of NVLink cabling and 5,000 connections.
However, this compact, high-powered architecture has proven challenging to cool effectively. Overheating GPUs can throttle performance, destabilise servers, and disrupt operations, leading NVIDIA to request multiple design changes from its suppliers.
In response to these issues, NVIDIA reiterated its collaborative approach.
“Nvidia is working with leading cloud service providers as an integral part of our engineering team and process,” a company spokesperson told Reuters, adding ‘Engineering iterations are normal and expected’.
While overheating concerns have dominated recent discussions, some industry experts suggest the problem may already be resolved. Semianalysis, a semiconductor-focused research firm, told Business Insider that NVIDIA suppliers implemented minor rack design changes months ago to address cooling issues. The firm’s chief analyst expressed confidence that while cooling could be an ongoing consideration, the immediate problem has been fixed.
Yet others, including Georgia Tech Professor Bara Cola, argue heat alone may not be the primary challenge. Cola, a thermal computing expert, suggested that ‘mechanical stress’ on components within the densely packed racks might be a more significant factor.
Written by Nicole Cappella Thu 21 Nov 2024