Why the B200 Blackwell chip will consolidate Nvidia's stranglehold ...
Nvidia has unveiled the B200 ‘Blackwell’, its latest artificial intelligence (AI) chip that can do some computational tasks 30 times faster than its current blockbuster, the H100 ‘Hopper’ — the chip that has helped the company gain a 80% market share.
The new chip, with even more computational power and optimised power consumption, will surely extend Nvidia’s dominance of this niche space.
Nvidia’s new Blackwell chip
The Blackwell graphic processing unit (GPU) has 208 billion transistors, compared with the 80 billion in the H100 that was launched last year, markedly increasing compute power. The new chip was “twice as powerful” when it came to training AI models, and had “five times their capability” in “inference” — the pace with which AI models such as Gemini or ChatGPT can generate responses — Nvidia chief executive Jensen Huang said at the chip’s launch at the company’s annual developer conference in San Jose, California, on Monday (March 18).
So, while training a version of the GPT model that powered ChatGPT (a 1.8 trillion-parameter model) would have previously taken 8,000 Hopper GPUs and 15 megawatts (MW) of electricity, the job can now be done by 2,000 new Blackwells while consuming just 4 MW of power, Huang said.
The company has said that its major customers including Google, Amazon, Microsoft, and OpenAI are expected to use the new chip in their cloud-computing services, as well as for their own AI products, the Financial Times reported.
“Blackwell offers massive performance leaps, and will accelerate our ability to deliver leading-edge models,” Sam Altman, CEO of OpenAI, said in a statement.
Leader of the GPU wave
Even before the latest announcement, Nvidia was already the third most valuable company in the US, behind only Microsoft and Apple. Shares of the Santa Clara-based company have surged nearly 250% over the past year, propelling it to the title of the world’s most valuable chipmaker, eclipsing storied competitors such as Intel and AMD.
“I hope you realise this is not a concert,” Huang quipped, half in jest, before he kicked off the developer conference from a hockey arena stage in the heart of Silicon Valley. The concert-like mood at the conference, which started on Monday and will run through Thursday, reflected the investor, developer, and media interest in the AI boom in general, and in Nvidia’s ability to supply the most crucial hardware to power this business — the GPU.
Nvidia has a stranglehold over these highly prized chips, which crunch data for AI models, and is likely to continue to dominate the global market for GPUs well into the foreseeable future.
Traditionally, the central processing unit (CPU) has been the most important component in a computer or server, and Intel and AMD dominated the market. GPUs are relatively new additions to the computer hardware market, and were initially sold as cards that plugged into a personal computer’s motherboard to add computing power to an AMD or Intel CPU.
Nvidia’s main pitch over the years has been that graphics chips can handle the computation workload surge of the kind that is needed in high-end graphics for gaming or animation applications far better than standard processors. AI applications too require tremendous computing power and have been progressively getting GPU-heavy in their backend hardware.
Most advanced systems used for training generative AI tools now deploy as many as half a dozen GPUs to every one CPU used, completely changing the equation in which GPUs were seen as add-ons to CPUs.
In its just published 2024 AI outlook, Moody’s Investors Service has said that “growing AI spending, model improvement and edge computing” will speed up AI adoption, and that AI investment will rise as firms move “from exploration to deployment”.
“A shortage of high-performance graphical processing units, essential for most AI computing, will persist in 2024, but supply will improve gradually”, Moody’s said.
Nvidia’s virtual monopoly over GPUs — which have the computing power and operational efficiency to run the calculations that allow AI companies working on LLMs (or large language models), such as ChatGPT or Gemini, to chomp down on massive volumes of data — has meant that the chipmaker is now swamped with orders that it is struggling to deliver.
The new B200’s promise of increased compute power could potentially mean a faster pathway to end these shortages.
