MediaTek Integrates Llama 3 Generative AI into Smartphone Chips

Mediatek, a leading semiconductor company, has recently integrated the cutting-edge open-source transformer-based generative AI model, Llama 3, into its Dimensity smartphone chips. This move marks a significant advancement in AI technology within mobile devices.

Meta, the parent company of Facebook, unveiled the Llama 3 Large Language Model (LLM) designed for various AI applications. With an impressive 8 billion parameters, Mediatek has successfully adapted this model to work seamlessly on the Dimensity 9300, Dimensity 8300, and upcoming Gen-AI enabled platforms.

The Dimensity 9300 stands out as the first smartphone chip in the market to feature the latest ARM Cortex-X4 and Cortex-A720 processors. Utilizing a unique architecture with four of each processor type and a custom AI processor unit (APU), this chip incorporates a hardware generative AI engine that accelerates transformer-based generative AI tasks by a factor of 8. Moreover, it leverages hardware-accelerated memory compression technology to efficiently manage the substantial memory requirements of transformer AI, enabling support for models with up to 33 billion parameters. Additionally, the Dimensity 9300 boasts LPDDR5T memory running at an impressive 9600Mbps, making it the fastest smartphone memory available today.

Building upon the foundation laid by Meta Llama 2, Mediatek's implementation of Llama 3 promises enhanced performance in terms of responsiveness, privacy, security, and reliability. This upgrade enables the chip to better understand language nuances, context, and handle complex tasks such as translation and dialogue generation with improved efficiency.

Llama 3's capabilities extend to effortlessly managing multi-step tasks while refining post-training processes to reduce false refusal rates, enhance response alignment, and diversify model answers. Furthermore, it significantly enhances functions like reasoning, code generation, text comprehension, instruction following, and overall world knowledge.

Developers keen on leveraging the power of Llama 3 can access it through MediaTek's NeuroPilot SDK. This software development kit serves as a comprehensive toolkit that facilitates and optimizes on-device GenAI inference capabilities across Mediatek's range of chips, catering to mobile platforms and edge-AI capable devices.

For more information on Mediatek's latest advancements and products, visit their official website at www.mediatek.com.