Kioxia offers memory search algorithm to reduce AI’s DRAM need

Memory maker Kioxia Inc. is offering its AiSAQ technology as open-source software to the design community claiming it will reduce generative AI’s requirements for DRAM.

If the technology is successful it could potentially boost the use of solid-state drives (SSDs) for AI applications and in consequence the use of NAND flash memory for which supply currently exceeds demand.

The software improves bothvector database scaling and the accuracy of retrieval-augmented generation (RAG) workflows and allows the direct use of solid-state drives (SSDs), the company said.

A technical paper on the approach – AiSAQ: All-in-Storage ANNS with Product Quantization for DRAM-free Information Retrieval – was published in April 2024 and the software is available via github at https://github.com/kioxiaamerica/aisaq-diskann

The ANNS, or approximate nearest neighbor search, is optimized to search for data in SSDs and delivers RAG without placing index data in DRAM – and instead searching directly on SSDs. RAG is a phase of AI that refines large language models (LLMs) with data specific to a company or application.

RAG identifies vectors that improve the model based on similarity between the accumulated and target vectors. For RAG to be effective, it retrieve the information most relevant to a query sufficiently rapidly.

Traditionally, ANNS algorithms are deployed in DRAM to achieve the high-speed performance required for these searches, the company said.

Kioxia claims that its AiSAQ technology is a scalable and efficient ANNS for billion-scale datasets with “negligible memory usage and fast index switching capabilities.”

“The Kiovia AiSAQ solution paves the way for almost infinite scaling of RAG applications in generative AI systems based on flash-based SSDs at the core,” said Axel Stoermann, CTO at Kioxia Europe GmbH. “Utilizing SSD-based ANNS, we are reducing the reliance on costly DRAM while matching the performance needs of leading in-memory solutions – enhancing the performance range of large-scale RAG applications significantly.”