Meta, hell-bent on catching up to its rivals in the generative AI space, is spending thousands of millions in the effort. A part of those billions goes to recruit AI researchers. But an even bigger chunk is being spent on hardware development, specifically chips to run and train Meta's AI models.
Meta unveiled the latest fruit of its chip development efforts, after Intel will announce its latest AI accelerator hardware. Called the “next-generation” Meta Training and Inference Accelerator (MTIA), the successor to last year’s MTIA v1, the chip runs models that include ranking and recommending display ads on Meta properties (e.g., Facebook).
The next generation MTIA is 5nm, in contrast to MTIA v1, which was developed for a 7nm process. (The "process" in chip manufacturing refers to the size of the smallest component that can be built on the chip in nanometers, nm.) The physical design of the next-generation MTIA is larger and features more processing cores than its predecessor. Although it consumes more power (90W vs. 25W), it also has more internal memory (128MB vs. 64MB) and runs at a higher average clock speed (1,35GHz vs. 800MHz).
The next-generation MTIA, according to Meta, is currently available in 16 data center regions and offers up to three times the overall performance of MTIA v1. If the “3x” statement seems ambiguous, you are not wrong. However, Meta claimed that the figure was obtained by evaluating the performance of "four key models" on both chips.
“We can achieve greater efficiency compared to commercially available GPUs because we control the entire stack,” Meta writes in a blog post.
First, Meta announces in the blog post that it is not currently using the next-generation MTIA for generative AI training workloads, despite the company stating that it has "several programs in place" to investigate. this matter. Second, Meta recognizes that the next-generation MTIA will complement rather than replace GPUs for running or training models.
Meta advances slowly, perhaps more than they expect.
Meta's AI teams are likely under pressure to drive down costs. By the end of 2024, the company is set to invest around $18 billion in GPUs to train and run generative AI models. With training costs for cutting-edge generative models running into the tens of millions of dollars, in-house hardware is an attractive option.
As Meta's hardware deteriorates, competitors are taking over, worrying Meta's leadership.
This week, Google introduced TPU v5p, its fifth-generation custom chip for training AI models, and Axion, its first model execution chip. Amazon has a variety of custom AI chip families. The year before, Microsoft debuted the Azure Maia AI accelerator and Azure Cobalt 100 CPU.
Meta claims in the blog post that "going from first silicon to production models" of the next-generation MTIA took less than nine months, which, to be fair, is shorter than the typical window between Google's TPUs. However, Meta must work hard to upgrade if it hopes to achieve some autonomy from third-party GPUs and keep up with its aggressive competition.
