Tenstorrent releases the vLLM TT plugin
Released the vLLM TT plugin. Using vLLM's out-of-tree platform plugin mechanism, it automatically registers Tenstorrent accelerators when the ttnn library is present, keeping the OpenAI-compatible API unchanged. Supports Llama 3.1–3.3, Qwen 2.5–3.6, Mistral, Gemma 3–4, DeepSeek V3 and multimodal variants, but the model implementations live in TT-Metal rather than in the plugin. The main differences from GPU serving are stage-based scheduling split into prefill-only and decode-only, single-process lane data parallelism for Galaxy systems, on-device sampling and asynchronous readback; speculative decoding, LoRA, prompt logprobs and multi-host serving are not supported. No performance figures are given.
Why it matters
Shows a path by which a non-mainstream accelerator joins the mainstream inference stack through a plugin that leaves the vLLM core untouched — the barrier to hardware diversification is coming down.
Source — Primary sources first; secondary reports are labelled “Press”
Officialvllm.aihttps://vllm.ai/blog/2026-09-07-vllm-tt-pluginAdded in Week 38, 2026 · See that week