What is the TurboQuant paper?

TurboQuant: Online Vector Quantization with Near-optimal Distortion Rate (arXiv:2504.19874), to be presented at ICLR 2026. It proposes a data-oblivious online vector quantizer aimed at KV-cache compression and vector search.

What is TurboQuant KV cache compression?

It quantizes key/value embeddings in decoder transformers so long-context inference uses less memory. The Google Research blog reports at least 6× KV-memory reduction in their tests, and the paper reports quality neutrality at 3.5 bits per channel on long-context benchmarks.

How do PolarQuant and QJL relate to TurboQuant?

TurboQuant is described as a two-stage design: a high-quality compression stage (PolarQuant / MSE path with random rotation + per-coordinate quantization) plus a 1-bit QJL residual correction for unbiased inner-product estimation.

Where can I read the primary sources?

Paper PDF: https://arxiv.org/pdf/2504.19874 — Google Research blog: https://research.google/blog/turboquant-redefining-ai-efficiency-with-extreme-compression/ — This site is unofficial and not affiliated with Google.

Back to home

Paper & method

TurboQuant paper explained

An independent, source-linked walkthrough of what the TurboQuant paper claims about KV-cache compression, PolarQuant, QJL, and the published benchmarks—not an official Google page.

What the paper is

TurboQuant is Google Research's online, data-oblivious vector quantization method for high-dimensional Euclidean vectors. The public paper is titled TurboQuant: Online Vector Quantization with Near-optimal Distortion Rate (arXiv:2504.19874), listed for ICLR 2026. Authors on the arXiv page include Amir Zandieh, Majid Daliri, Majid Hadian, and Vahab Mirrokni.

The work targets two practical bottlenecks: LLM KV-cache memory and high-dimensional vector search (inner-product / ANN style workloads).

How it works (plain language)

Per the Google Research blog and paper abstract, TurboQuant is usefully thought of as a two-stage compressor:

Main compression (PolarQuant / MSE path). Randomly rotate vectors, then apply strong per-coordinate quantization. PolarQuant (companion work, AISTATS 2026; arXiv:2502.02617) is described as converting Cartesian coordinates into a polar “shorthand” so expensive per-block normalization overhead can be reduced.
Residual correction (QJL). A small residual budget (described as 1 bit in the blog) applies Quantized Johnson–Lindenstrauss (QJL) so inner-product / attention-score estimates stay unbiased.

The paper argues the resulting distortion rates sit close to information-theoretic lower bounds (within a small constant factor, ≈2.7 in the abstract). That theory story is why searchers looking for turboquant algorithm / quantization often land on the paper itself—this page is only a map.

Cited results (from primary sources only)

Figures below are quoted from the Google Research blog (2026-03-24) or the paper abstract / LongBench table. This site did not re-run the benchmarks.

KV memory: blog — “reducing the key value memory size by a factor of at least 6×” on needle-in-a-haystack style long-context tests.
Bit-width / quality (paper abstract): “absolute quality neutrality with 3.5 bits per channel and marginal quality degradation with 2.5 bits per channel” for KV-cache quantization.
Attention speed (blog): “4-bit TurboQuant achieves up to 8× performance increase over 32-bit unquantized keys on H100 GPU accelerators” for computing attention logits.
LongBench (paper Table 1, Llama-3.1-8B-Instruct): Full Cache average 50.06; TurboQuant at 3.5-bit average 50.06; TurboQuant at 2.5-bit average 49.44.
Training: blog — can quantize the KV cache “without requiring training or fine-tuning.”

For RAM / VRAM planning searches (turboquant ram, turboquant llm), treat these as paper/blog reference points—not a promise for your model, hardware, or fork.

Primary links

Paper PDF — arxiv.org/pdf/2504.19874
PolarQuant — arXiv:2502.02617
Google Research blog — TurboQuant announcement

Next: running it locally

Paper ≠ product binary. Community and engine integrations change quickly. See the how-to / status guide for MLX, LM Studio, llama.cpp forks, vLLM, and GitHub pointers—or return to the TurboQuant overview.

FAQ

What is the TurboQuant paper?: TurboQuant: Online Vector Quantization with Near-optimal Distortion Rate (arXiv:2504.19874), to be presented at ICLR 2026. It proposes a data-oblivious online vector quantizer aimed at KV-cache compression and vector search.
What is TurboQuant KV cache compression?: It quantizes key/value embeddings in decoder transformers so long-context inference uses less memory. The Google Research blog reports at least 6× KV-memory reduction in their tests, and the paper reports quality neutrality at 3.5 bits per channel on long-context benchmarks.
How do PolarQuant and QJL relate to TurboQuant?: TurboQuant is described as a two-stage design: a high-quality compression stage (PolarQuant / MSE path with random rotation + per-coordinate quantization) plus a 1-bit QJL residual correction for unbiased inner-product estimation.
Where can I read the primary sources?: Paper PDF: https://arxiv.org/pdf/2504.19874 — Google Research blog: https://research.google/blog/turboquant-redefining-ai-efficiency-with-extreme-compression/ — This site is unofficial and not affiliated with Google.