Groq Secures $640m Funding for Inference AI Chip Development

AI inference chip developer Groq has recently secured an impressive $640 million in funding, valuing the company at $2.8 billion. This significant investment will fuel Groq's ambitious plans to deploy over 108,000 Language Processing Unit (LPU) Inference Engine chips, which are manufactured by GlobalFoundries, by the end of the first quarter of 2025.

With a rapidly expanding community of over 360,000 developers leveraging its GroqCloud platform, Groq is at the forefront of enabling the creation of AI applications using openly-available models such as Llama 3.1 from Meta, Whisper Large V3 from OpenAI, Gemma from Google, and Mixtral from Mistral. The funding will be instrumental in enhancing the capacity of Groq's tokens-as-a-service (TaaS) offering and introducing new models and features to the GroqCloud ecosystem.

The LPU, a crucial component in the AI infrastructure, is seamlessly integrated into data centers alongside CPUs and Graphics Processors, facilitating both training and inference processes. Customers have the flexibility to opt for on-premise deployment or access the platform via APIs. Currently, Groq is achieving remarkable performance with its Llama-2 70B model, processing over 300 tokens per second per user.

The Series D funding round was spearheaded by funds and accounts managed by BlackRock Private Equity Partners, with significant contributions from existing investors and newcomers like Neuberger Berman, Type One Ventures, and strategic investors including Cisco Investments, Global Brain’s KDDI Open Innovation Fund III, and Samsung Catalyst Fund. Samir Menon, Managing Director at BlackRock Private Equity Partners, expressed confidence in Groq's potential, stating, "The market for AI compute is meaningful, and Groq's vertically integrated solution is well positioned to capitalize on this opportunity."

Jonathan Ross, CEO and Founder of Groq, emphasized the critical role of inference compute in powering AI advancements, stating, "We intend to democratize access to cutting-edge AI products, making them available to a broader audience beyond just the major tech players." With the additional funding secured, Groq is set to deploy more than 100,000 additional LPUs into GroqCloud, further expanding its capabilities. Ross also highlighted the company's commitment to talent acquisition, aiming to significantly enhance its workforce to support its growth trajectory.