vLLM Semantic Router releases d3-ultra: a 27B multimodal decision model that answers every question in a single read
MarkTechPost reported on October 26, 2026 that the vLLM Semantic Router team released d3-ultra, a 27B multimodal decision model under Apache-2.0. It does not generate text: a single read of the input returns a probability for each candidate answer to every question, covering routing, classification, bounding boxes and video checks.
What changed
MarkTechPost reported on October 26, 2026 that the vLLM Semantic Router team released d3-ultra, a 26.09B-parameter dense multimodal decision model (including a 0.46B vision encoder), with weights available under the Apache-2.0 license, part of its Decision 3.0 family (0.6B to 27B). Unlike generative models, d3-ultra does not output text: the caller submits a state (text or JSON, plus images or video) and a set of typed questions—Choice, Yes/No, Score, Box, Track—and the model reads the input once, returning a probability for each candidate answer to each question. The report says the configuration shows a Qwen3.5 backbone with 64 layers (48 linear-attention layers plus 16 full-attention layers). The model card does not name the base checkpoint, while the vLLM recipe says the sibling d3 was trained on Qwen3.8-27B, so d3-ultra's own base origin remains unclear. Each question has a 131,072-token context; over-long requests fail outright with max_length_exceeded rather than being silently truncated. The Box type always returns one box and a confidence, and a confidence close to 0 is treated as the target not existing. The report lists deployment paths including: Transformers (requires transformers==5.17.0) with an optional flash-linear-attention kernel, the vllm-sr serve command merged via PR #5449 on October 25, 2026, a hosted interface the model card says offers a free key, and an official Hugging Face Space demo. On performance, the team compared it with GLiDE V1.2, d1-ultra, d3-turbo, d3 and Jev 1.14 on two private test sets it built: SR-Text covers 10,411 requests and 73,495 questions (opportunity-corrected accuracy), where d3-ultra scored 87.2 to lead, above GLiDE V1.2's 85.1 and sibling d3-turbo's 83.1; only in the "art and human taste" domain did d3-ultra score 89.6 against d1-ultra's 90.8, a difference the report says is within the margin of error. SR-Box covers 5,678 box questions on 4,984 images, scored at IoU 0.5; d3-ultra scored 58.2% across 10 domains and 54.6% across 9 domains, below d3-turbo's 60.0% and 59.6%. The team says d3-turbo is stronger on natural images while d3-ultra leads on charts. On latency, the report says testing used AutoModel with BF16 weights on a single AMD Instinct MI325X: a single question on a single photo took 304 ms, 255 questions took 2.16 s, and each additional question added about 7.3 ms on average. No official quantized version has been listed. All benchmark numbers are the team's self-reported results on private test sets, and public indexes show "submitted, not yet listed."
What it means for developers
The model's shape matters more for integration than for any single capability. Conventional generative APIs bill by output token and plan capacity by concurrency, whereas d3-ultra's cost and latency structure is "one forward pass per input, with flatter increments as questions grow"—the reported 7.3 ms per question suggests that when several judgments are merged into one call, the main cost sits in that single read. For teams already using a semantic router to dispatch requests, this offers an alternative path: instead of prompting a general LLM to output JSON and then parsing it for judgments like "which model should this request go to," "which queue does this ticket belong to" or "is this content violating policy," take probabilities over controlled, typed questions. The output is inherently structured probability values, removing a layer of parse-failure risk. The trade-off is flexibility: the model does no free generation, question types and options must be designed in advance, and tasks outside those types cannot be done with it. On licensing, a distinction matters: the report says weights are released under Apache-2.0, which is a license term rather than merely downloadable weights; in principle this favors commercial use and redistribution, but actual use should still check the model card and the license file in the repository, and license rights should not be inferred from weight availability. On deployment, 26.09B dense parameters running in BF16 need substantial VRAM; the reported test was done only on a single AMD MI325X and no official quantized version exists yet, so teams wanting to self-host on smaller-VRAM devices currently lack an official path. The two key capability figures point in opposite directions—text judgments lead the family across the board, while visual box selection trails d3-turbo—so model choice cannot be a blanket "bigger and newer is better" and must be weighed per workload. The base origin remains an information gap: the configuration shows a Qwen3.5 architecture, the recipe only says sibling d3 is based on Qwen3.8-27B, and the model card does not name d3-ultra's base checkpoint, which affects judgment about training-data provenance, license inheritance and room for later fine-tuning. All benchmarks are self-reported on the team's private test sets and public indexes have not yet listed it, so 87.2 and 58.2% should be treated as vendor-reported until third-party reproduction.
Integration and evaluation
The following is conditional analysis and advice based on the report, not verified fact. First, if your team is using a generative LLM for request routing or classification, start with a same-input comparison: take a batch of real requests from existing routing logs, run them through both the current prompting approach and d3-ultra's Choice or Yes/No question form, and compare accuracy and end-to-end latency against the same set of human-labeled answers. The trigger condition is that decision categories are limited and enumerable and answers can be written as fixed options; if judgments often require free explanation or multi-step reasoning, replacement is not suitable. Measurable metrics are accuracy on the same labeled set and P95 end-to-end latency, and a switch should be considered only if d3-ultra is no worse than the current approach and total latency is lower. Second, if the workload is mainly visual localization, do not default to d3-ultra just because SR-Text leads. The report says d3-turbo is stronger on natural-image box selection and d3-ultra leads on charts, so stratify by your own image types (natural photos and charts separately), measure both models' accuracy at IoU 0.5, and separately track the miss rate where "returned confidence is near 0 but the target actually exists," then decide by the share of each image type; d3-ultra's localization advantage may only pay off when chart-type images are a large share. Third, do VRAM and throughput accounting before self-hosting: estimate VRAM needs for 26.09B dense BF16 plus the vision encoder and KV cache, test on target hardware whether answering 255 questions in a single request is near the reported 2.16 s, and treat the "304 ms per single question" as the first-read overhead for a capacity baseline; if VRAM is insufficient and no official quantized version exists, mark that path as not currently feasible rather than quantizing on your own and assuming quality is unchanged. Fourth, design questions with the 131,072-token hard limit as a constraint: long documents or multi-frame video need chunking first, and instrument max_length_exceeded errors separately so request failures are not miscounted as low confidence. Fifth, the model card says a hosted interface offers a free key, but the report does not describe quotas, stability or data terms; verify the service terms yourself before production use.
Limits and open questions
The accuracy figures in the report are all self-reported by the team on private test sets, not reproduced by third parties, and public indexes are marked "submitted, not yet listed." Latency data is a different kind of information: the report says it was measured on a single AMD Instinct MI325X with AutoModel and BF16 weights, not a private benchmark, and behavior on other hardware and inference stacks is unknown; the report also does not publish comparative latency against other models on the same hardware. The report does not say whether the two private test sets were constructed in a way that favors the task types this family is good at. There is an information gap on base origin: the configuration shows a Qwen3.5 architecture, the recipe only says sibling d3 is based on Qwen3.8-27B, and the model card does not name d3-ultra's base checkpoint, so the Qwen-family backbone claim cannot be treated as confirmed fact. Deployment testing is limited to a single AMD Instinct MI325X and BF16 weights, with no official quantized version yet. The hosted interface's free-key terms, limits and availability are not covered in the report, and such arrangements may change at any time. As of this catalog check (October 27, 2026), d3-ultra does not appear in KimiSeek's available model catalog; the API availability described in the report refers to the vendor's own interface and is unrelated to this site. Readers concerned about available models and billing should rely on the site's pricing page.