Nemotron 3.5 Super VL Lands on SageMaker JumpStart: Model Claims 1M Context, Endpoint Caps at 15,600 Tokens
An AWS blog post dated November 9 announced that NVIDIA's Nemotron 3.5 Super VL is available on SageMaker JumpStart in two variants, NVFP4 and BF16. It has 120B total parameters with 12B active, and the weights are provided under the OpenMDW 1.1 license. However, the hosted endpoint's service configurations limit context to between 520 and 15,600 tokens.
What changed
According to an AWS blog post published on November 9, 2026 (authors Venu Kanamatareddy, Evan Kravitz, and Naidile Murali), NVIDIA's Nemotron 3.5 Super VL is now listed on Amazon SageMaker JumpStart. As described in the post, the model is a vision-language model in the Nemotron 3.5 series that accepts image, video, and text input and outputs text. Its language backbone has 120B total parameters with 12B activated per token: 88 layers alternating Mamba-2 and MoE (512 routed experts, 22 activated per token), interspersed with 8 attention layers. The backbone is initialized from Nemotron 3 Super Base and distilled from Nemotron 3 Ultra Base, and the vision side uses a C-RADIO encoder. JumpStart offers two variants: NVFP4, obtained from BF16 through quantization-aware distillation, is about 75GB, runs on G7 instances with an RTX PRO 4500 Blackwell, and executes with 4-bit weights and activations; BF16, the full-precision reference weights, is about 249GB and runs on 8 H200 GPUs in ml.p5en.48xlarge. The weights are provided under the OpenMDW License Agreement 1.1 and support English, French, German, Italian, Japanese, Spanish, and Chinese. The blog says the model supports up to 1 million tokens of context, but the JumpStart service configurations only go up to 520 to 15,600 tokens; the checkpoint includes a multi-token prediction head, but JumpStart service configurations do not enable speculative decoding, and the endpoint does not load LoRA adapters.
What it means for developers
The first impact for developers is the interface shape. According to the blog, the JumpStart endpoint exposes a vLLM-driven chat completions interface, with reasoning on by default; thinking content is placed in the message's reasoning field and returned separately from content. This means existing clients that parse only a single content field will need changes, otherwise they will drop the body text or mistake the reasoning process for the body. Media must be sent inline as base64 data URLs, because the endpoint is network-isolated and cannot fetch URLs; this significantly increases request body size and makes approaches like "give it a link and let the model download it" fail outright. The second impact is capacity planning. The model card says up to 1 million tokens of context, but the hosted endpoint's service configurations actually provide only 520 to 15,600 tokens, and each request accepts only one image and one video. A 1,536×1,024 page image consumes roughly 1,536 tokens, and the blog explicitly says the model accepts images rather than raw PDFs, so each page must first be rendered as an image. It follows that a PDF of several dozen pages cannot be sent in one pass on this hosted path; it must be requested page by page, or multiple pages must be combined into one image to fit the context budget. The third impact is latency expectations. The blog's measurements are that under the generate configuration (250 input tokens, 1,500 output tokens, 5 concurrent streaming requests), NVFP4 on G7 delivers about 63 tokens/sec per request with about 0.45s to first token; but the same g7.24xlarge under the summarize configuration with 10,000 input tokens takes 12.62s to first token. Image and video requests usually fall on the long-prompt side, so before planning to use this for an interactive GUI agent, it is worth measuring this first-token magnitude yourself. On quality, the blog cites five multimodal benchmarks from NVIDIA's model card (with reasoning enabled): MMLongBench-Doc 62.82 vs. 62.82, OCRBench v2 English 68.12 vs. 68.34, CharXiv reasoning 80.82 vs. 79.88, Video-MME v2 32.85 vs. 32.54, and OSWorld-Verified 51.14 vs. 52.30, with the first figure being BF16 and the second NVFP4. The blog says the evaluation settings were kept consistent and only the checkpoint differed. These are vendor-reported numbers with no third-party reproduction seen.
Integration and evaluation
The following three are conditional suggestions based on reported facts, not verified conclusions. First, decide the variant. The reported change is that JumpStart offers both NVFP4 and BF16 checkpoints, with NVFP4 as the default and starting on ml.g7.24xlarge, and BF16 only on ml.p5en.48xlarge. The choice to make: for pure inference serving, start with NVFP4; only if you need SFT, reinforcement learning, distillation, domain adaptation, or custom quantization should you download the BF16 reference weights and self-host. The key premise that would change this choice is fine-tuning needs: the blog explicitly says the endpoint provides only base weights and does not load LoRA adapters, so if your plan depends on hot-swapping LoRA, the hosted path simply does not work. A testable criterion is to run both checkpoints on the same batch of your own document screenshots and video clips, compare the difference in task accuracy and JSON output compliance, and set the acceptable difference to no more than the error level corresponding to your existing human review rate. Second, decide the service configuration before discussing context. The reported change is that inference_config_name determines model length: generate 2,275, interact 520, summarize 13,650, modify 15,600 tokens, and the blog's image, video, and reasoning examples all use modify. In terms of choice, use modify for requests containing images, video, or multi-step reasoning, and generate for short text extraction, and estimate how many pages each request can handle based on "about 1,536 tokens per page." The premise is that multi-page documents must be split into multiple requests or combined into one image. A testable criterion is to measure, before launch, the rejection rate for over-length requests and the p95 time to first token, and to treat the 12.62s reference value at 10,000 input tokens as a lower-bound comparison for the latency budget. Third, decide the reasoning switch. The reported change is that reasoning is on by default, thinking content is returned separately, and video requests with reasoning are recommended to have max_tokens set to at least 8,192. In terms of choice, use enable_thinking=False for tasks that need short, direct replies, such as ticket field extraction and image captioning, and keep it on for multi-step analysis and GUI grounding. The premise for changing the choice is whether the task really needs intermediate steps. A testable criterion is to compare, on the same batch of samples, field extraction accuracy, output token count, and end-to-end latency with reasoning on and off, to see whether the saved tokens come at an acceptable loss of precision. Before deployment, also confirm ml.g7 quota (zero initially for new accounts) and SageMaker Python SDK version 3.25 or higher, and pass the instance type explicitly, otherwise the SDK falls back to ml.m5.large and the build fails.
Limits and open questions
Nemotron 3.5 Super VL is not currently in KimiSeek's model catalog; the reported availability refers to AWS SageMaker JumpStart, a vendor-side hosted service, not a model already listed on this site. Other unknowns should be preserved as well: the five benchmarks cited above come from NVIDIA's model card and are vendor-reported; the blog itself notes that the NVFP4 and BF16 performance numbers were run on different GPU systems and configurations, not an isolated precision comparison, so one cannot directly infer quantization loss from 63 tokens/sec versus 140.85 tokens/sec. The 1 million token context claimed by the model card is not exposed on the hosted endpoint, and the actual limit depends on the service configuration; video frame sampling is fixed when the endpoint starts, and the endpoint also rejects passing media_io_kwargs per request, so per-request control of sampling requires self-hosting. The weights are provided under the OpenMDW License Agreement 1.1; open weights are not the same as an open-source license, and the terms for commercial use, redistribution, and derivative models require reading the license text. The source gives no pricing; costs depend on the chosen instance, per the AWS SageMaker pricing page. Although Chinese is on the supported language list, the source provides no benchmark data for Chinese scenarios.
Illustrations from the original report






