ARM Introduces Ethos U85 to Boost Embedded Transformer AI Models

Embedded algorithms have been evolving rapidly, transitioning from traditional Digital Signal Processing (DSP) to Convolutional Neural Networks (CNN) and now to transformers. According to Paul Williamson, senior vice president and general manager of the IoT business at ARM, the Ethos U85 was specifically designed to cater to the needs of transformers. These advanced transformer models are currently being utilized for various applications such as object recognition, detection, pose detection, and language models.

The Ethos U85 accelerator core is a groundbreaking innovation that is paired with the high-end Cortex M85 microcontroller core in the Corstone 320. This combination serves as a pre-verified virtual model, facilitating chip and software developments right from the initial stages. Williamson emphasizes the importance of looking ahead by at least 5 years, foreseeing the increasing adoption of AI technologies. He envisions AI making its way into devices like smart cameras, revolutionizing industries with its capabilities.

One of the key features of the Ethos U85 is its ability to address the growing demand for video as a sensor. This versatile platform can be utilized for a wide range of applications, including fault inspection on production lines, shelf monitoring in warehouses, and person monitoring in smart homes. Unlike its predecessors, the U55 and U65, which were optimized for RNN and CNN, the U85 excels in handling transformer models without relying on the CPU for processing, resulting in a significant boost in system performance.

Williamson sheds light on the innovative design of the U85, highlighting the redesign of MAC units to accommodate dynamic weightings and a memory system that eliminates the need to copy weights back to main memory or the CPU for calculations. This streamlined approach not only enhances efficiency but also paves the way for seamless integration of AI capabilities into various applications. The shift towards smaller language models (SML) is gaining traction, with ARM successfully running a 7 billion parameter model on a Cortex A510 core, demonstrating impressive reading speeds.

Looking towards the future, ARM's partners are already exploring the possibilities of deploying genAI on edge models to test before silicon, with expectations of seeing the U85 integrated into devices by 2025. The focus is on leveraging smaller library models to enhance support and implementing transformer networks for defect detection. As the industry continues to embrace AI technologies, the Ethos U85 accelerator core stands at the forefront, driving innovation and transforming the way we interact with intelligent systems.