Underdog Releases Saluki 27B: A 7.89GB 2-bit Qwen3.8-27B That Beats the Original on Tool Calling
According to an October 9, 2026 MarkTechPost report, Conway Research's Underdog released Saluki 27B under Apache 2.0, compressing the 54GB Qwen3.8-27B to a 7.89GB 2-bit mixed GGUF that runs on stock llama.cpp; its tool-calling score beats the original, but competition math and multi-step reasoning drop by 12 to 18 points.
What changed
According to an October 9, 2026 MarkTechPost report, Underdog, the on-device assistant project under Conway Research, released Saluki 27B under the Apache 2.0 license. The model is a 2-bit mixed-precision GGUF version of Qwen3.8-27B, with a file size of 7.89GB, versus 54GB for the full BF16 model. The report says its construction stacks three layers of work: the base is the Qwen team's 27B dense model Qwen3.8-27B (64 layers, hybrid Gated DeltaNet linear attention and gated attention, native 262,144 tokens); on top of that is ISTA-DASLab's GSQ-RCO GGUF, whose smallest file, IQ2_XS, is 8.4GB at an average of 2.50 bits per weight; Underdog then applies its own compression and tool-calling optimization, producing a file named IQ2-mix marked with imatrix, though the report notes Underdog has not published the full recipe for this step. On the runtime side, the report says the model runs directly on stock llama.cpp and applications built on it, supports full GPU offload, and has optional vision add-ons of 629MB or 928MB. On benchmarks, the report gives two sets of results that Underdog itself states: in one comparison within the same framework as the full model, Underdog Bench (120 tasks from BFCL v4, thinking mode off, temperature 0) gives Saluki 88 versus 84 for the full model; parallel tool calls 42 versus 35; and on SWE-bench Verified, 30 of 50 issues fixed versus 33. In comparisons against published full-size scores, IFEval is 93.5 versus 91.5, IFBench 72.7 versus 71.0, MBPP+ 78.0 versus 83.9, MuSR 67.5 versus 79.6, AIME 2025 79.2 versus 96.7, and AIME 2026 80.0 versus 94.6. The report also lists ISTA GSQ-RCO IQ2_XS and PrismML Bonsai 2 27B as comparisons, and stresses that each vendor uses its own test framework and that cross-vendor scores are not directly comparable; Bonsai 2 is smaller but requires PrismML's llama.cpp fork.
What it means for developers
For developers who need local agents, the substantive change in this news is this: a 27B-class tool-calling model whose weights quantize to under 8GB, and which loads without a third-party llama.cpp fork. This directly changes judgments about VRAM and deployment thresholds—full offload on a single consumer GPU or even a large-memory machine becomes realistic, and it makes a hybrid architecture of a local agent loop plus a remote model fallback more calculable in cost. But open weights must be distinguished from license rights: the report says the file is released under Apache 2.0, which is a license-level statement, and whether commercial use is allowed should still be checked against the model card and the license text itself; downloadable weights do not by themselves equal a clear scope of authorization. More attention is needed to the fact that capabilities are unevenly distributed: the report says tool-calling and instruction-following metrics are close to or slightly above the original, while math and multi-step reasoning metrics such as AIME and MuSR fall by 12 to 18 points. This means the quantized version suits tool orchestration, function selection, and format-constrained workloads, but is not suited to being treated as the main reasoning engine. In addition, all scores come from Underdog or vendor self-tests it cites, and different vendors use different test frameworks, so they are reported claims rather than independent reproductions, and cross-model rankings should be treated as incomparable. The report does not disclose bits per weight, context length, or the compression recipe, and these unknowns will affect real-world performance under long context and batched concurrency.
Integration and evaluation
Recommendation one (tool-calling routing): whether to use Saluki 27B as the function-selection and parameter-generation layer of a local agent depends on whether your workload is mainly parallel tool calls and structured output. If your scenario involves single-turn or few parallel calls with strict response formats, the report says its parallel tool-calling and instruction-following metrics are close to or above the original, so running it locally can be a priority; if your workload depends on long chains of multi-step planning, the report says MuSR and AIME drop noticeably, so you should keep the original or a stronger model as fallback. Measurable evaluation criteria: run the same frozen batch of tasks against your own tool schema, comparing Saluki and the original (or your current online model) on tool-name selection accuracy and parameter JSON validity, plus task completion rate in parallel-call scenarios. Recommendation two (compression scheme selection): choosing among candidates in the same 2-bit range depends on whether you are willing to maintain a non-stock runtime. The report says Saluki runs on stock llama.cpp, while PrismML Bonsai 2 requires its own llama.cpp fork, which directly affects build, upgrade, and CI complexity; at the same time, the report says Saluki does not disclose bits per weight or context, while the comparison scheme discloses 2.50 bits per weight and file size. If your team does not want to fork the inference framework, you can treat 'loadable on stock llama.cpp' as a hard prerequisite, then compare size and accuracy; if size is the primary constraint, you need to accept the fork-maintenance cost. Measurable evaluation criteria: on the same target machine, record first-token latency, peak VRAM, and stability under long context (close to your actual maximum input length), and verify the llama.cpp version required to load the file. Recommendation three (post-quantization acceptance): whichever side you choose, make 'can it still complete end-to-end tasks after quantization' rather than benchmark scores the go-live threshold. Measurable evaluation criteria: using a sample of your real production traffic, compare the number of passed fix-type tasks, tool-call rounds, and failure retry rate, and set an explicit fallback threshold, for example automatically falling back when the end-to-end task pass rate falls below some acceptable proportion of the original. The above is conditional analysis and recommendations, not statements of fact from the report.
Limits and open questions
First, it must be made clear: the Saluki 27B in the report (a 2-bit GGUF derivative of Qwen3.8-27B) does not currently appear in KimiSeek's model catalog; the distribution method described in the report is obtaining GGUF weights through channels such as Hugging Face and running them locally on stock llama.cpp, and does not involve API availability on the KimiSeek platform—any statement about API availability refers to the publisher's own channels. Second, all benchmark numbers in the text are relayed from Underdog's self-tests and the published scores it cites; they are publisher claims, and this article has not independently reproduced them; the report itself also notes that each vendor uses its own test framework and that cross-vendor scores are not directly comparable. Third, the report does not disclose the file's bits per weight, context length, or the full recipe for the third layer of compression, so performance under long context, batched concurrency, and different hardware remains unknown. Fourth, that the weights are released under Apache 2.0 is a license statement in the report; whether specific commercial conditions are met must be based on the model card and the license text, and availability of the weights does not equal a clear scope of authorization. Finally, the base capability statements for Qwen3.8-27B (64 layers, hybrid attention, 262,144 native context) likewise come from that report and have not been verified by this article.