Fractile licenses RISC-V vector processor IP for AI accelerator

UK startup Fractile has licensed a 64bit RISC-V vector processor for its AI accelerator chip from Andes Technology.

Fractile, with chip design in London and Bristol, is developing AI inference accelerators based on in-memory compute, and aim to be able to run large language, vision and audio AI models up to 100 times faster than existing hardware at a tenth the cost.

Fractile will use the ACE (Andes Automated Custom Extension) for in memory computing along with the Andes Domain Library and the AX45MPV RISC-V vector processor for its first generation data centre AI inference accelerator. The 64bit in-order dual-issue 8-stage CPU core (above) has 1024bit Vector Processing Unit (VPU) with high bandwidth vector local memory (HVM) and Andes CoDense extension for further compaction of code size. ACE is separately licensable for customized scalar and vector instructions.

• Fractile raises $15m for AI in memory compute
• NATO backs in memory compute AI chip startup

Model inference – the process of serving these trained models – is coming to be the dominant portion of compute costs, exceeding the cost of model training, says Fractile. It is developing novel circuits to execute 99.99% of the operations needed to run model inference in on-chip memory. This removes the need to shuttle model parameters to and from processor chips, instead baking computational operations into memory directly.  This architecture drives both much higher energy efficiency as well as dramatically improved latency on inference tasks (tokens per second per user in an LLM context, for instance).

“The AX45MPV, with strong compute capabilities, high memory bandwidth and the flexible ACE tool, has been chosen by innovative AI companies large and small since its debut in 2023,” said Dr. Charlie Su, President and CTO of Andes Technology. “Andes RISC-V vector processors have enabled many AI SoCs to break free from architecture limitation and achieve new levels of performance and efficiency.”

“The limitations of existing hardware present the biggest barrier to AI performance and adoption. Andes Technology has unmatched technical and commercial leadership on RISC-V vector processors and is a natural partner for us as we build Fractile’s accelerator systems. Building hardware for AI acceleration is intrinsically hard – the world’s leading models can change overnight, while chips take time to bring to market. Software-programmable vector processors like Andes’ are a key part of staying robust to these changes,” said Dr. Walter Goodwin, CEO and founder of Fractile.