← All model news

Baidu opens ERNIE 5.5 Flash Base weights: 280B MoE, context extended to 1M, and three things to measure first

According to a Jiqizhixin (Machine Heart) report on November 5, 2026, Baidu released and opened the weights of the pretrained base model ERNIE 5.5 Flash Base: sparse MoE with 280B total parameters and 14B activated per token, pretrained on 33.5T tokens, a 128K training context extended to 1M during fine-tuning and inference, plus FP8 and BF16 weight versions and vLLM and SGLang recipes.

Published Updated

What changed

On November 5, 2026, Jiqizhixin reported that Baidu released a new base model, ERNIE 5.5 Flash Base, and opened its weights. The reported specifications are: sparse MoE, 280B total parameters, 14B activated per token, pretrained from scratch on 33.5T tokens, training context up to 128K, extended to 1M tokens during fine-tuning and inference; a 43-layer Transformer with hidden size 4096 and 64 attention heads, where every layer except the first uses MoE blocks in its feed-forward network. The weights are published on Hugging Face in FP8 and BF16 versions, and the vLLM and SGLang communities provided corresponding cookbooks. The report focuses on three architecture changes. Attention mixes MLA and HCA: of the 43 layers, only 6 are MLA layers retaining token-level full-context access, while the other 37 are heavily compressed attention (HCA) that compresses every 128 tokens into a single cache entry; the MLA layers further add sparse attention DSA, using a lightweight indexer to pick the top-k highest-scoring historical positions for each query, which the report says matches dense attention in pretraining loss. The MoE layers use Multi-Router, with several independently initialized sigmoid routers scoring at the same time, 512 routed experts per block, selecting 10 plus 1 shared expert each time, with tokens first projected into a 2048-dimensional latent space for computation and then mapped back. On the training objective, ECHO no longer attaches an external MTP module; instead, the last layer of the backbone also handles future-token prediction and doubles at inference as a built-in draft for speculative decoding. There are further changes such as mHC residual, a MultiMax output head, and a roughly 200K BPE vocabulary (with digits split in groups of three); the FlashMask-4 operator for large head dim has been open-sourced in the PaddlePaddle repository. The report says that in official comparisons, ERNIE 5.5 Flash Base leads DeepSeek-V4-Flash Base, Hy3-preview-Base and MiMo-V2.5-Base overall on knowledge, reasoning, math, code, multilingual and long-context benchmarks; in the item-by-item comparison with DeepSeek-V4-Flash-Base (284B parameters, 13B activated), the code gap is the largest, with HumanEval+ at 82.3 versus 69.5, LiveCodeBench at 40.1 versus 31.9, and FullStackBench at 54.6 versus 43.3, while it trails slightly on a small number of evaluations such as ChineseSimpleQA. The report also says the official release of ERNIE 5.5 Flash is expected soon.

What it means for developers

This is an opening of weights, not the launch of a directly callable service. The report contains no information on an API launch, hosted pricing or commercial terms, and does not state the license type; open weights and an open-source license are two different things, and whether it can be used commercially or redistributed has to be checked against the license text in the model card or repository. Releasing the weights means the follow-up direction is handed to the user: this is a Base model, which the report says already has complete general capabilities, but turning it into a product that can hold conversations or run agents requires attaching your own instruction tuning, reinforcement learning or domain adaptation. For adoption and deployment, the most practical effects are in two places. First, the cost structure. HCA compresses the KV of most layers into one entry per 128 tokens, and only 6 MLA layers retain per-token access; the reported design intent is to lower long-context pre-fill and KV cache usage. If you estimate memory and concurrency from dense-attention experience, you may well get it wrong; but the actual behavior of compressed attention on fine-grained retrieval also needs to be verified on your own tasks and cannot be inferred directly from "long-context benchmarks are broadly solid". Second, the inference path. ECHO lets the last backbone layer double as the draft; the report says this adds almost no parameters and achieves a higher acceptance rate than an external MTP, but whether this speedup is enabled in the inference stack you use, and how it is configured, depends on the implementation in the vLLM/SGLang recipes rather than taking effect automatically from the model itself. The 280B total parameter count also means that even with only 14B activated per token, the weights themselves still need multi-GPU memory; at a rough estimate of about one byte per parameter in FP8, the weights alone are on the order of several hundred GB, an account to settle before deployment. On competition, the report places ERNIE 5.5 Flash Base in the Flash tier and names DeepSeek-V4-Flash Base, Hy3-preview-Base and MiMo-V2.5-Base as same-tier rivals. For developers, one more selectable base is a good thing, but all benchmarks in the report are officially self-reported, and the specific versions of the comparison targets cannot be confirmed in this check; their value is directional rather than absolute.

Integration and evaluation

The following are conditional suggestions and inferences, not reported facts. Decision one: start with FP8 or BF16 weights. The corresponding reported fact is that Baidu released both FP8 and BF16 versions. If your goal is to serve externally right away and your target cards support FP8 well, you can start with FP8 to save memory and gain throughput; if you still need to continue pretraining, do heavier fine-tuning, or your hardware has limited FP8 support, start from BF16. The fork is "inference only" versus "still training". A testable criterion: pick a set of your own representative inputs, run the same evaluation suite on both weight versions, record how consistent the task metrics are along with peak memory, first-token latency and throughput, quantify the quality loss, and then decide, rather than assuming FP8 is lossless. Decision two: is 1M context a selling point or a trap. The corresponding reported facts are a 128K training context and extension to 1M during fine-tuning and inference. If your workload is long-document QA, repository-level code, or the context of a long-running agent, you can run a length-ladder test first and then decide the ceiling; if it is only short dialogue or single-turn tasks, planning context within 128K is more prudent. The fork is whether you really need to fit far more than 128K into one request. A testable criterion: run one of your own retrieval or QA sets at each of 32K, 128K, 256K and 1M, record accuracy, first-token latency and KV memory as they change with length, find the point where accuracy drops noticeably, and set the ceiling there. Decision three: use the Base now for self-training, or wait for the official release. The report says the official release is expected soon but gives no date or specifications, so this is essentially a choice between "existing data and training pipeline" and "waiting for an out-of-the-box Chat/Agent model". The condition is whether you have domain data at scale and post-training experience; if not, self-training from the Base may cost more than it returns. A testable criterion: run a small-scale trial with your existing SFT data, compare against your current production model on a held-out domain set, and see whether the gain reliably covers the replacement and operations cost. If you want to keep tracking it, three places are worth watching: the license text on the model card, the actual configuration items for long context and speculative decoding in the vLLM/SGLang recipes, and whether the official release gives evaluation criteria consistent with the Base.

Limits and open questions

The information in this article comes largely from the vendor's side: the architecture description, routing and attention design, the effect of ECHO, and the item-by-item comparison with models such as DeepSeek-V4-Flash Base all come from official data relayed in the report and were not independently reproduced here. The versions of the rival models named in the report are also outside the scope of what can be verified, so the benchmark numbers should be treated as self-reported; moreover, what is compared are Base models, and conclusions cannot be directly swapped with the Chat versions each vendor has already launched. On licensing, the report only says the weights are open and gives no license name or commercial and redistribution terms; being able to download the weights does not mean they can be used freely, and the model card and repository must be checked before deployment. The report also does not mention an API or hosted service. ERNIE 5.5 Flash Base is not currently in the KimiSeek model catalog; the download locations and inference recipes mentioned here all point to Baidu and the vLLM and SGLang communities, and any availability described in the report is the provider's account; if hosted services or APIs appear later, their availability and pricing should be governed by the provider and this site's pricing page. In addition, 1M is the extension range for fine-tuning and inference, and the report does not state quality at that length; "six long-context benchmarks are solid" does not mean the same holds on your tasks. The official release has not shipped yet, and its capabilities and the scope of what is opened remain unknown.

Three minutes to your first working call

Create an account and an API key. Calls use your account balance. 30% off your first top-up. Top up ¥500, pay ¥350, receive ¥500 in credit. Top up ¥1000, pay ¥850, receive ¥1000 in credit You can still start with a ¥10 trial top-up.

Sources and editorial process

刚刚,百度开源全新预训练基模:文心 5.5 Flash Base 主打效率

AI assisted the analysis and translation of the cited sources, followed by automated quality checks. Recommendations are not test results. Consult the current catalog for availability and pricing.