FuriosaAI Introduces Energy-Efficient AI Processor at Hot Chips

The latest breakthrough in AI technology comes in the form of Furiosa's revolutionary RNGD processor, a Tensor Contraction Processor designed for high-performance large language model (LLM) and multimodal model inference. This cutting-edge chip, implemented in TSMC's advanced 5nm manufacturing process, boasts impressive specifications that are set to redefine the landscape of AI computing.

Operating at a clock frequency of 1.0GHz, the RNGD processor delivers exceptional performance metrics. With a BF16 data type, it achieves a remarkable 256TFLOPS, while pushing boundaries further to reach 512TFLOPS at FP8 and 512TOPs with INT8 data type. Equipped with 256Mbytes of on-chip SRAM and the capability to be linked to 48Gbytes of external HBM3 DRAM, the chip offers a bandwidth of 1.5Tbytes per second, setting a new standard for AI processing power.

SemiFive, a key player in the semiconductor industry, has played a pivotal role in bringing Furiosa's RNGD processor to market. The collaboration between the two companies has resulted in the successful testing of the RNGD processor with large language models such as GPT-J and Llama 3.1. Impressively, a single RNGD PCIe card can achieve throughput performance of 2,000 to 3,000 tokens per second, depending on the context length of models with approximately 10 billion parameters.

One of the standout features of the RNGD PCIe card is its energy efficiency, boasting a thermal design profile (TDP) of just 150W. This is a stark contrast to the over a kilowatt power consumption required by GPU-based solutions, making the RNGD processor a sustainable and accessible AI computing solution that aligns with the industry's growing emphasis on green computing.

June Paik, co-founder and CEO of FuriosaAI, expressed confidence in the RNGD processor, stating, "RNGD is a sustainable and accessible AI computing solution that meets the industry's real-world needs for inference." The strategic partnership with Supermicro further enhances the impact of Furiosa's technology, enabling Supermicro systems to achieve significant reductions in power consumption per card while maintaining exceptional inference performance.