How do I use TurboQuant today?

There is no single official Google binary. As of mid-2026 public signals: vLLM has merged TurboQuant KV-cache support (see v0.20.0 release notes); Apple Silicon users often look at community MLX packages; llama.cpp work lives mainly in community forks; LM Studio support is still a feature request until a backend ships it.

Does LM Studio support TurboQuant?

Not as a finished first-party feature at the time we last checked. Open requests exist (e.g. lmstudio-bug-tracker #1719 for TurboQuant+ / llama.cpp flags, and mlx-engine #296). LM Studio typically inherits support when llama.cpp or its MLX engine ships it—re-check those issues before assuming UI toggles exist.

Is there a TurboQuant MLX package?

Yes—community repositories such as github.com/manjunathshiva/turboquant-mlx advertise an MLX implementation (PyPI name often listed as turboquant-mlx-full). Treat install/convert commands as README-owned; they change.

Where is TurboQuant on GitHub?

Google Research published the paper and blog; community code appears under multiple orgs/forks (vLLM upstream, MLX packages, llama.cpp forks such as TheTom/llama-cpp-turboquant). Search GitHub for “turboquant” and verify stars/activity/README before installing. Avoid confusing unrelated “turboquant” trading repos.

Back to home

Local run status

How to use TurboQuant (status guide)

A practical, unofficial map of where TurboQuant shows up for local and server inference—MLX, LM Studio, llama.cpp forks, vLLM, and GitHub—without inventing install steps.

Status matrix (high level)

Runtime	Status	How to verify
vLLM	Merged upstream (PR #38479, 2026-04-15); called out in the v0.20.0 release notes as TurboQuant KV-cache support.	Check current vLLM docs / `--kv-cache-dtype` help for TurboQuant presets on your installed version.
MLX (Apple Silicon)	Community package exists (manjunathshiva/turboquant-mlx). Not “Google official.”	Follow that repo's README for install/convert/serve.
llama.cpp	Community forks such as TheTom/llama-cpp-turboquant (and related TurboQuant+ work). Mainline merge status has been contested in public trackers—do not assume stock llama.cpp binaries include it.	Read the fork README for build flags / cache-type options; confirm against the commit you actually build.
LM Studio	Feature requests open; not a finished first-party toggle when last checked.	Watch lmstudio-bug-tracker #1719 and mlx-engine #296 (or newer successors).
Google official code drop	Public face remains the paper + research blog. Do not expect an “official APK” or Google Play binary named TurboQuant.	Use /paper for primary citations.

vLLM (server path)

If you already run vLLM ≥ the release that includes PR #38479 / the TurboQuant notes in v0.20.0, look for KV-cache dtype presets related to TurboQuant in that version's CLI/docs (preset names have evolved across review iterations). Example discovery pattern:

# On the machine where vLLM is installed — verify locally
python -m vllm.entrypoints.cli.main --help 2>/dev/null | rg -i 'kv-cache|turbo'
# or read the release notes / docs for your exact version

We deliberately do not hard-code a single flag string here: PR discussions show preset renames. Copy from the docs that match your wheel.

MLX / Apple Silicon

Search interest in turboquant mlx / mlx turboquant is high. A concrete community project is manjunathshiva/turboquant-mlx, which describes weight + KV-cache compression for MLX and points to a PyPI distribution. Install and convert flows belong in that README (including any python -m turboquant_mlx.convert … examples they publish).

LM Studio's MLX engine tracker has an open TurboQuant request (#296)—shipping there depends on engine/backends, not on this website.

llama.cpp, TurboQuant+, and “llama turboquant”

Queries like llama turboquant and turboquant plus usually mean community llama.cpp work (for example forks advertising turbo3/turbo4 KV cache types). Those flags are fork-specific until (or unless) they land in the binary you actually run.

Practical check: clone/build the fork you trust → read its README for --cache-type-k / --cache-type-v (or successor) options → confirm with --help on that build. Do not paste flags from screenshots into an unrelated stock binary.

Models & GitHub hygiene

turboquant models: TurboQuant is primarily a KV-cache / vector quantizer in the research framing—not a new foundation model family. “Models” usually means “which HF/GGUF/MLX checkpoints work with a given integration.”
turboquant github: start from the paper citation, then verify engine PRs/repos. Unrelated older projects also use the name “turboquant” (e.g. trading bots)—check descriptions before pip install.
how to use turboquant: (1) read the paper page, (2) pick a runtime from the matrix above, (3) follow that runtime's docs for your GPU/Apple Silicon setup.

FAQ

How do I use TurboQuant today?: There is no single official Google binary. As of mid-2026 public signals: vLLM has merged TurboQuant KV-cache support (see v0.20.0 release notes); Apple Silicon users often look at community MLX packages; llama.cpp work lives mainly in community forks; LM Studio support is still a feature request until a backend ships it.
Does LM Studio support TurboQuant?: Not as a finished first-party feature at the time we last checked. Open requests exist (e.g. lmstudio-bug-tracker #1719 for TurboQuant+ / llama.cpp flags, and mlx-engine #296). LM Studio typically inherits support when llama.cpp or its MLX engine ships it—re-check those issues before assuming UI toggles exist.
Is there a TurboQuant MLX package?: Yes—community repositories such as github.com/manjunathshiva/turboquant-mlx advertise an MLX implementation (PyPI name often listed as turboquant-mlx-full). Treat install/convert commands as README-owned; they change.
Where is TurboQuant on GitHub?: Google Research published the paper and blog; community code appears under multiple orgs/forks (vLLM upstream, MLX packages, llama.cpp forks such as TheTom/llama-cpp-turboquant). Search GitHub for “turboquant” and verify stars/activity/README before installing. Avoid confusing unrelated “turboquant” trading repos.

Back to the TurboQuant overview or the paper explainer.