← All model news

LCVN: From a Starting Image and a Single Instruction to a Full Navigation Route, the Two Routes Involve Different Trade-offs

According to a Machine Heart (Jiqizhixin) report on November 8, 2026, LCVN proposes language-conditioned visual navigation: given only a first-person image of the starting point and an English instruction, the model predicts a complete action sequence with no environment feedback for correction along the way. The dataset contains 39,016 trajectories and 117,048 human-verified instructions. The paper compares a world-model-plus-policy route with a unified-model route, reporting in-domain success rates of 43% and 42%, and 19% and 27% in unfamiliar off-road environments.

Published Updated

What changed

According to a Machine Heart (Jiqizhixin) report dated November 8, 2026, a project called LCVN proposes a language-conditioned visual navigation task: the only inputs are one first-person image of the starting point and one English instruction, and the model must predict the full action sequence in one pass, including planar displacement and heading changes. The key setting is that there is no real environment feedback for correction during prediction; the model can only keep rolling out along its own predicted states, and the final evaluation is on real robot trajectories, so what is assessed is offline trajectory prediction. The report says the work was accepted as a NeurIPS 2026 Oral. The accompanying dataset contains 39,016 trajectories and 117,048 manually verified instructions. The trajectories come from Go Stanford, ReCon, SCAND, HuRoN and TartanDrive; the first four are used for training and in-domain evaluation, while TartanDrive's off-road scenes test generalization in unfamiliar environments. Each trajectory has three kinds of English instruction (concise, detailed, and landmark-based); 640 were randomly sampled for human evaluation, with overall quality rated 5.2/6.0. The paper compares two routes: LCVN-WM works together with a policy model, LCVN-AC, and the policy learns entirely in the world model's latent space; LCVN-Uni, built on Anole-7B, encodes images, language and discretized actions into a unified token sequence and gives both the next action and the next-frame observation in a single prediction. The report gives in-domain validation success rates of 43% and 42%, 19% and 27% in the unfamiliar TartanDrive environment, and 36% and 34% on the mixed test set; for 8-step continuous prediction, SSIM@8 is 0.293 and 0.218, with the latter better on DreamSim. An ablation shows that removing the language condition from LCVN-WM lowers in-domain success from 43% to 35%. In a failure case the paper provides, LCVN-Uni got the turn direction right but did not generate the corridor and the target white door that should appear after the turn, so subsequent actions lost reliable visual grounding.

What it means for developers

Judging by the reported numbers, developers first need to distinguish the metrics: task success is defined as the predicted endpoint being closer to the real goal than that data source's average step length. This means different data sources are judged by different rulers, success rates across datasets cannot be compared directly, and in scenes with larger steps the same error is more likely to count as success. Second, the advantages of the two routes do not overlap. The reported in-domain success rate is slightly higher for world model plus policy, whereas the unified model is higher in unfamiliar off-road environments, while the world model is better at the structural and temporal coherence of continuous frame prediction; the report itself stresses that visual prediction ability and navigation ability need to be examined separately. This has practical implications for integration: if the planning module consumes predicted frames or latent states, frame metrics matter as much as success rate; if only the final trajectory is used, frame metrics can be set aside. Third, the language condition is not decoration. The reported ablation shows a clear drop in in-domain success when the instruction is removed, indicating that the instruction plays the role of goal description in the model, and instruction quality should be treated as a first-class variable rather than a prompt detail. Finally, these conclusions all rest on offline prediction, and the report explicitly lists closed-loop validation on real robots as work still to be done; treating this as an online navigation policy that can go straight onto a robot exceeds what the report covers.

Integration and evaluation

A workable evaluation order is: first reproduce the reported in-domain setting on the public code and dataset, confirming that success rate, trajectory error and frame metrics all line up, then carve out a holdout set from your own scenes and recompute with the same success definition. If your site's average trajectory step length differs substantially from the paper's data sources, the success rate will be recalibrated, and you should then also report absolute endpoint error rather than looking only at success; next, run a language-condition ablation, running instruction and no-instruction versions on the same trajectories to see whether the gap is close to the reported 8 percentage points. The following two points are conditional suggestions and recommended judgments, not reported facts. First, route choice should depend on whether the target domain falls inside the training distribution: if your scenes are close to the four indoor data sources, you can try the world-model-plus-policy combination first; if the target is off-road, unmapped or clearly unfamiliar terrain, you can try the unified model first. This choice flips when you need continuous frame prediction to support downstream decisions, because the reported SSIM@8 for the unified model is lower. The premise is that you are able to do out-of-domain holdout evaluation on your own data; a testable criterion is whether the success-rate difference in your scenes is consistently in the same direction across multiple random seeds, rather than a single run. Second, whether to build language instructions into your data pipeline depends on whether you can produce instructions matching the three predefined styles: if you only have free-form spoken or non-English instructions, you should not assume you will get the same benefit. Testable criteria are the success-rate gap between instruction-style-aligned and unaligned groups on the same holdout trajectories, and whether the gain from instructions versus no instructions is reproducible.

Limits and open questions

The report provides code and a dataset; it does not say whether model weights are released, nor does it cover inference or commercial licensing terms, so public code cannot be equated with usable or commercially usable weights. The task itself is limited to offline trajectory prediction; closed-loop execution and recovery after errors have not been validated. Instructions cover only three predefined English styles, and more natural phrasing and multilingual instructions remain untested. The paper's failure case also shows that once frame prediction goes astray, subsequent actions lose reliable grounding. LCVN is currently not in KimiSeek's model catalog, and the report mentions no hosted API; if related interfaces appear later, their availability, quotas and terms are up to the publisher and should be confirmed against official documentation. The report also mentions that the team is running a WorldNav track at the RoboWorld Challenge 2026 using the same mixed test set, and participating teams should check whether the leaderboard's criteria match the paper's.

Three minutes to your first working call

Create an account and an API key. Calls use your account balance. 30% off your first top-up. Top up ¥500, pay ¥350, receive ¥500 in credit. Top up ¥1000, pay ¥850, receive ¥1000 in credit You can still start with a ¥10 trial top-up.

Sources and editorial process

NeurIPS 2026 Oral|LCVN:一张起始画面、一段语言指令,能否预测完整的导航路线?

AI assisted the analysis and translation of the cited sources, followed by automated quality checks. Recommendations are not test results. Consult the current catalog for availability and pricing.