Google Research outlines algorithms that may ease AI memory squeeze

Google Research said it has created a set of algorithms that could reduce the need for a massive amount of memory for AI workloads.

In a blog post, Google Research outlined TurboQuant, a compression algorithm that reduces memory overhead for vector quantization. Vectors are the way AI models understand and process information. Small vectors describe simple attributes while high-dimensional vectors capture complex information. The high-dimensional vectors are powerful but lead to bottlenecks in key-value cache.

To date, the solution to enhance vector search and vector quantization has been to throw more memory at the problem. Of course, there isn't enough memory capacity and shortages that plague the tech sector. Google's algorithm news dinged shares of storage and memory companies. Micron Technology’s recent earnings were stellar.

• Memory price surge may crimp on-premise AI, data center deployments
• IBM overhauls FlashSystem with AI, proprietary memory, SLA focus
• Storage becomes more strategic amid AI buildout

Google said TurboQuant is a compression algorithm designed to address memory overhead in vector quantization. Google also outlined two other algorithms--Quantized Johnson-Lindenstrauss (QJL), and PolarQuant--that are used by TurboQuant. "In testing, all three techniques showed great promise for reducing key-value bottlenecks without sacrificing model performance," said Amir Zandieh, Research Scientist, and Vahab Mirrokni, VP and Google Fellow, Google Research.

Key points:

• TurboQuant is a compression algorithm that optimally addresses the challenge of memory overhead in vector quantization.
• QJL shrinks complex, high-dimensional data while preserving the essential distances and relationships between data points. The algorithm creates a high-speed shorthand that requires zero memory overhead.
• PolarQuant addresses the memory overhead problem by using polar coordinates. This allows LLMs to skip the data normalization step because it maps data onto a fixed, predictable circular grid where the boundaries are already known.

"As AI becomes more integrated into all products, from LLMs to semantic search, this work in fundamental vector quantization will be more critical than ever," said Zandieh and Mirrokni.