Jiuzhang Cloud Releases Alaya-DFlash2 Speculative Decoding for GLM-5.3, Claims 3.41x Decode Speedup at 32k Context
According to an InfoQ report on October 20, 2026, Jiuzhang Cloud released Alaya-DFlash2, a speculative decoding acceleration scheme for GLM-5.3 that includes a parallel-drafting draft model and a path selection mechanism, with weights published. The vendor's own evaluation claims single-concurrency decode speed at 8xH200, TP=8 is 3.41x that of native autoregressive decoding, and that accepted lengths across 1k to 32k low-entropy contexts are higher than two comparison schemes. These numbers and comparisons are vendor self-reported benchmarks and await third-party reproduction.
What changed
According to an InfoQ report on October 20, 2026, Jiuzhang Cloud launched Alaya-DFlash2, a speculative decoding acceleration scheme for GLM-5.3, composed of a draft model capable of parallel drafting and a path selection mechanism, with weights published under the name AlayaNeW/GLM-5.3-DFlash2; the report did not specify concrete open-source license terms. The report says the scheme is based on Jiuzhang Cloud's AlayaNeW intelligent computing engine and trains a companion draft model around GLM-5.3. On evaluation, the report gives data under an 8xH200, TP=8 environment: single-concurrency decode speed is about 288.2 tokens/s, versus about 84.5 tokens/s for native autoregressive, a 3.41x factor; corresponding total output throughput rises from 78.95 to 236.47 tokens/s. As concurrency rises, the advantage narrows: at 4-concurrency the throughput is 2.24x that of native autoregressive, and at 64-concurrency the throughput is 608.71 tokens/s, a 44.9% improvement over native autoregressive at 420.13 tokens/s. The report also lists low-entropy context accepted lengths: 1k, 2k, 8k, 16k and 32k are 4.74, 4.97, 5.28, 5.50 and 5.39 respectively, all higher than the IncoAI-DFlash2 and RedHat-DSpark it compares against; it also says accepted length leads in 9 of 11 task categories, with code and role-play categories close to IncoAI-DFlash2.
What it means for developers
The practical meaning of speculative decoding for developers is that it changes the cost structure of generation on the same hardware, not the model capability itself. Under the report's framing, draft accepted length staying around 5 in long-context scenarios means one main-model forward pass can verify and pass about 5 candidate tokens, reducing the number of repeated main-model decode rounds and directly compressing the wait for continuous output after the first token. For teams running their own GLM-5.3 inference service, this amounts to trading for decode throughput without changing the main model or its output distribution, at the cost of loading an extra draft model's weights and the associated VRAM and scheduling overhead. The report also shows the gain varies with concurrency: the single-concurrency speedup is the largest, while at 64-concurrency total throughput improvement narrows to 44.9%, indicating the benefit is more concentrated in low-concurrency interactive scenarios, with relatively limited per-unit throughput gains in high-concurrency serving. It is worth noting that all numbers come from vendor self-testing, the test environment is fixed at TP=8 and 8xH200, accepted length depends on the specific data distribution and may differ outside low-entropy contexts; and published weights do not equal clear license terms, so license information in the repository still needs to be checked before commercial use.
Integration and evaluation
It is advisable to make trade-offs under the following conditions rather than directly adopting the evaluation conclusions. The first decision is whether to introduce this draft model. If the inference load is mainly long-context, low-to-medium concurrency work such as long-document QA and RAG summarization, the reported accepted lengths above 5 at 16k to 32k support running a real-traffic comparison test; if the load is mainly high-concurrency short requests, the reported 64-concurrency throughput gain of only 44.9% means you should first estimate whether the extra VRAM footprint and scheduling cost of the draft model are worthwhile. The evaluation criterion can be: measure actual accepted length and inter-token latency on your own 32k-length samples; if accepted length is clearly below the reported level or the added VRAM lowers the concurrency ceiling, do not adopt it. The second decision is the configuration of number of draft positions and verification batch. The report shows acceptance rate in later draft positions is still notable (at 32k, 49.43% for position 5 and 29.51% for position 8), which supports setting verification depth deeper than the default; but this benefit holds only if the backend acceptance rate also holds on your own data. If acceptance rate decays quickly on task types where the report did not lead, such as code and role-play, increasing draft length would only waste compute. The evaluation criterion can be: tally pass rate by position, find the position where acceptance rate falls below a set threshold and truncate draft length accordingly. All of the above are conditional suggestions based on the reported data, not deployment conclusions provided by the vendor.
Limits and open questions
All performance data in the report comes from Jiuzhang Cloud's self-testing, with no third-party reproduction provided, nor disclosure of test datasets, prompt templates and decoding parameters; accepted length depends heavily on data distribution and may not reproduce on your own traffic. The evaluation environment is fixed at 8xH200, TP=8, and behavior under other hardware and parallel configurations is unknown. The report's claim of open source provides only a weights address and does not state a license type; weights being available does not mean the license terms permit arbitrary commercial use or redistribution, and the repository declaration should be authoritative. In addition, Alaya-DFlash2 is an independent draft model scheme for GLM-5.3 and is a separate matter from the main model itself; GLM-5.3 does have a corresponding entry in the current KimiSeek catalog, but the Alaya-DFlash2 acceleration scheme itself is not listed as a callable model on KimiSeek, and relevant availability is subject to the vendor and the weights repository.
Illustrations from the original report
