← All model news

NVIDIA's TensorRT LLM VisualGen speeds up FLUX 3 Image Klein: 3.46x text-to-image and 4.45x image-to-image on one GPU, with cache and sparse attention costing LPIPS

According to an NVIDIA developer blog dated November 6, 2026, TensorRT LLM VisualGen gives a deployment configuration for FLUX 3 Image Klein: on a single B200, 2048×2048 text-to-image is up to 3.46x faster and image-to-image 4.45x faster than uncompiled BF16, at the cost of quality shifts from NVFP4, caching and sparse attention.

Published Updated
Cover from the original report — NVIDIA's TensorRT LLM VisualGen speeds up FLUX 3 Image Klein: 3.46x text-to-image and 4.45x image-to-image on one GPU, with cache and sparse attention costing LPIPS
Cover from the original reportOriginal report ↗

What changed

NVIDIA published a deployment tutorial on its developer blog on November 6, 2026, explaining how to configure Black Forest Labs' FLUX 3 Image Klein in the VisualGen runtime of TensorRT LLM. According to the post, the model uses Qwen3-VL-4B-Instruct for text conditioning, a diffusion Transformer to refine image latents, an autoencoder to decode and, for image-to-image, to encode reference images; it offers Base and distilled checkpoints, and VisualGen supported BF16, FP8 and NVFP4 variants at release. The post provides a container image, YAML configuration, offline inference scripts, and a way to expose an OpenAI-compatible /v1/images/generations endpoint with trtllm-serve. The reported test conditions are a single B200 at 2048×2048; Base uses 50 denoising steps with CFG 3.5, while the distilled version uses 16 steps without CFG. Relative to uncompiled BF16 inference, the combined optimizations cut text-to-image from 22.36 seconds to 6.46 seconds (3.46x) and image-to-image from 54.64 seconds to 12.28 seconds (4.45x); on a single RTX PRO 6000 Blackwell Server Edition the post reports 1.73–3.25x. The latency table shows denoising accounts for 96.8% of total time for Base text-to-image and 84.0% for the distilled version, and 97.8% and 89.0% for image-to-image, which is why the post focuses optimization on the diffusion Transformer.

What it means for developers

For integrators, the key point of this tutorial is that it ties speed and image quality into one configurable trade-off. For teams preparing to put FLUX 3 Image Klein into design tools or image editors, the most direct change is more choice of inference stack: the same checkpoint can run in BF16, FP8 or NVFP4, and can be combined with quantized attention, sparse attention, step caching and multi-GPU parallelism. It is important to distinguish that the post gives LPIPS shifts for quantized attention, sparse attention and Cache-DiT, but reports no image-quality change data for the FP8 precision tier or multi-GPU parallelism; the quality budget can only be calculated for the items with data, and one should not assume all optimizations lose fidelity equally. In other words, performance and quality budgets must be set together, not by launching on the fastest configuration and fixing quality later. Second, the reported bottleneck distribution determines the optimization order: in the single-B200 pipeline timings, denoising takes the vast majority of time, while text encoding and decoding plus post-processing take only a few hundred milliseconds, so optimizing prompt handling or tokenization first yields far less than changing the diffusion Transformer's precision and attention. Third, the OpenAI-compatible endpoint makes integration cost close to a normal image API, but this endpoint is provided by the TensorRT LLM runtime, so deployment, GPU memory and operations remain on your side; it is not the same as a hosted service. Fourth, prompt upsampling is not included in the reported latency, so if the product pipeline relies on BFL's upsampler, end-to-end time must be measured separately.

Integration and evaluation

The following are conditional suggestions based on what the post reports, not verified conclusions. Suggestion one: choose the precision tier by whether you can accept an LPIPS shift, not by GPU model. The post reports NVFP4 is up to 1.62x (text-to-image) and 1.58x (image-to-image) faster than BF16 on Blackwell, but in the example the NVFP4 step already produces an LPIPS shift of 0.044 relative to BF16. If the workload is batch draft generation followed by human selection, you can default to NVFP4 and use the share of every 100 images rejected by humans as an acceptance metric; if output goes directly to customer delivery or must match existing assets stylistically, build a BF16 baseline first, then enable optimizations one at a time and record LPIPS and human review results at each step. This assumes the hardware is within the Blackwell data-center cards or RTX PRO 6000 Blackwell Server Edition that the post explicitly focuses on; the numbers do not apply to other hardware. Suggestion two: whether to enable caching depends on whether you use Base or the distilled version. At a threshold of 0.24, the post reports Cache-DiT nearly halves NVFP4 Base text-to-image latency (1.97x at 1024×1024, 1.94x at 2048×2048), but for the 16-step distilled version only 1.26x and 1.20x, with corresponding LPIPS of 0.038, 0.036 and 0.033, 0.022. The condition is concrete: with Base and a goal of interactive previews the gain is large; if the distilled version already brings a single image down to a few seconds, the relative gain is small and you take on an additional quality shift. Measurable criteria are whether the acceptance rate on the same prompt batch is not below an agreed proportion of the uncached version, and whether single-image P95 latency actually drops, not just the average. Suggestion three: multi-GPU only pays off when denoising dominates. The post assigns Base's positive and negative branches to different GPU groups and then splits sequences with Ulysses; from 1 to 8 GPUs on DGX B200 it cuts 2048×2048 text-to-image median latency from 14.90 seconds to 3.00 seconds, with denoising itself 6.01x faster; the distilled version goes from 2.73 seconds to 1.08 seconds, but the post itself notes that about half the time in the eight-GPU distilled version is outside denoising, limiting end-to-end gains. A measurable criterion is to test the end-to-end P95 latency ratio for the same requests on your own prompt set: if the 1-to-8-GPU end-to-end speedup is clearly below the reported denoising speedup, non-denoising stages already take too large a share and adding GPUs is not worthwhile; first check at low resolution whether CUDA Graphs can cover kernel launch overhead before deciding to scale out. Two easy-to-miss configuration details should also be settled before integration: VC-Attention requires the container to set TLLM_FMHA_LIBS=+prims_ts, otherwise the relevant kernels are unavailable; Sol-Attention's disabled_until_timestep specifies a noise level rather than a step count, and the post says 0.96 corresponds under the packaged scheduler to the first 12 dense steps of the 50-step Base and the first 4 steps of the 16-step distilled version, while 16 dense steps requires setting 0.94; misconfiguring it directly changes the balance of quality and speed.

Limits and open questions

First, the availability boundary: FLUX 3 Image Klein and its Base and distilled checkpoints are not currently in KimiSeek's model catalog, with no exact match; the OpenAI-compatible image endpoint and trtllm-serve serving method mentioned in the post are both provided by the TensorRT LLM runtime on the vendor side, and their availability, versions and lifecycle are unrelated to KimiSeek's model list, so before integration refer to the original post and vendor releases. Second, the speedup ratios and LPIPS numbers in the post come from experiments with different conditions and cannot all be given the same preconditions. The single-GPU pipeline latency in Table 1 is explicitly a single B200 at 2048×2048 with NVFP4 plus VANILLA attention and torch.compile, averaged over 10 requests after two warm-ups, and that statistical basis applies only to that table; the end-to-end 3.46x (text-to-image) and 4.45x (image-to-image) are likewise single-B200, 2048×2048 results, while the 1.73–3.25x on RTX PRO 6000 Blackwell Server Edition is another platform. The multi-GPU data comes from DGX B200 scaling from 1 to 8 GPUs and reports median latency for 2048×2048 text-to-image; the CUDA Graphs results of 2.25x and 2.16x are a separate experiment with eight B200s at 1024×1024 and VANILLA attention. The caching section reports results at both 1024×1024 and 2048×2048, and the source does not restate the statistical basis and hardware configuration for every experiment individually. So changing hardware, resolution, GPU count or concurrency method may invalidate these numbers. Third, the experiments use seed 0 by default, the prompts are summarized, and the reported latency excludes prompt upsampling; LPIPS only measures perceptual similarity to the corresponding baseline, not edit-instruction following, text-rendering accuracy or product acceptance criteria, so your own task set still needs separate evaluation. Fourth, the container tags in the post include branch names and development version numbers, so confirm image and kernel version stability before production use. Finally, this tutorial covers only inference configuration and does not discuss weight license terms; the checkpoints are available on Hugging Face, but whether commercial use and redistribution are allowed depends on the model card and license text, and open weights are not the same as an open-source license.

Illustrations from the original report

Source illustration 1 — NVIDIA's TensorRT LLM VisualGen speeds up FLUX 3 Image Klein: 3.46x text-to-image and 4.45x image-to-image on one GPU, with cache and sparse attention costing LPIPS
Source illustration 1Original report ↗
Source illustration 2 — NVIDIA's TensorRT LLM VisualGen speeds up FLUX 3 Image Klein: 3.46x text-to-image and 4.45x image-to-image on one GPU, with cache and sparse attention costing LPIPS
Source illustration 2Original report ↗
Source illustration 3 — NVIDIA's TensorRT LLM VisualGen speeds up FLUX 3 Image Klein: 3.46x text-to-image and 4.45x image-to-image on one GPU, with cache and sparse attention costing LPIPS
Source illustration 3Original report ↗
Source illustration 4 — NVIDIA's TensorRT LLM VisualGen speeds up FLUX 3 Image Klein: 3.46x text-to-image and 4.45x image-to-image on one GPU, with cache and sparse attention costing LPIPS
Source illustration 4Original report ↗
Source illustration 5 — NVIDIA's TensorRT LLM VisualGen speeds up FLUX 3 Image Klein: 3.46x text-to-image and 4.45x image-to-image on one GPU, with cache and sparse attention costing LPIPS
Source illustration 5Original report ↗
Source illustration 6 — NVIDIA's TensorRT LLM VisualGen speeds up FLUX 3 Image Klein: 3.46x text-to-image and 4.45x image-to-image on one GPU, with cache and sparse attention costing LPIPS
Source illustration 6Original report ↗
Source illustration 7 — NVIDIA's TensorRT LLM VisualGen speeds up FLUX 3 Image Klein: 3.46x text-to-image and 4.45x image-to-image on one GPU, with cache and sparse attention costing LPIPS
Source illustration 7Original report ↗
Source illustration 8 — NVIDIA's TensorRT LLM VisualGen speeds up FLUX 3 Image Klein: 3.46x text-to-image and 4.45x image-to-image on one GPU, with cache and sparse attention costing LPIPS
Source illustration 8Original report ↗
Source illustration 9 — NVIDIA's TensorRT LLM VisualGen speeds up FLUX 3 Image Klein: 3.46x text-to-image and 4.45x image-to-image on one GPU, with cache and sparse attention costing LPIPS
Source illustration 9Original report ↗
Source illustration 10 — NVIDIA's TensorRT LLM VisualGen speeds up FLUX 3 Image Klein: 3.46x text-to-image and 4.45x image-to-image on one GPU, with cache and sparse attention costing LPIPS
Source illustration 10Original report ↗
Source illustration 11 — NVIDIA's TensorRT LLM VisualGen speeds up FLUX 3 Image Klein: 3.46x text-to-image and 4.45x image-to-image on one GPU, with cache and sparse attention costing LPIPS
Source illustration 11Original report ↗
Source illustration 12 — NVIDIA's TensorRT LLM VisualGen speeds up FLUX 3 Image Klein: 3.46x text-to-image and 4.45x image-to-image on one GPU, with cache and sparse attention costing LPIPS
Source illustration 12Original report ↗
Source illustration 13 — NVIDIA's TensorRT LLM VisualGen speeds up FLUX 3 Image Klein: 3.46x text-to-image and 4.45x image-to-image on one GPU, with cache and sparse attention costing LPIPS
Source illustration 13Original report ↗
Source illustration 14 — NVIDIA's TensorRT LLM VisualGen speeds up FLUX 3 Image Klein: 3.46x text-to-image and 4.45x image-to-image on one GPU, with cache and sparse attention costing LPIPS
Source illustration 14Original report ↗
Source illustration 15 — NVIDIA's TensorRT LLM VisualGen speeds up FLUX 3 Image Klein: 3.46x text-to-image and 4.45x image-to-image on one GPU, with cache and sparse attention costing LPIPS
Source illustration 15Original report ↗
Source illustration 16 — NVIDIA's TensorRT LLM VisualGen speeds up FLUX 3 Image Klein: 3.46x text-to-image and 4.45x image-to-image on one GPU, with cache and sparse attention costing LPIPS
Source illustration 16Original report ↗
Source illustration 17 — NVIDIA's TensorRT LLM VisualGen speeds up FLUX 3 Image Klein: 3.46x text-to-image and 4.45x image-to-image on one GPU, with cache and sparse attention costing LPIPS
Source illustration 17Original report ↗
Source illustration 18 — NVIDIA's TensorRT LLM VisualGen speeds up FLUX 3 Image Klein: 3.46x text-to-image and 4.45x image-to-image on one GPU, with cache and sparse attention costing LPIPS
Source illustration 18Original report ↗
Source illustration 19 — NVIDIA's TensorRT LLM VisualGen speeds up FLUX 3 Image Klein: 3.46x text-to-image and 4.45x image-to-image on one GPU, with cache and sparse attention costing LPIPS
Source illustration 19Original report ↗
Source illustration 20 — NVIDIA's TensorRT LLM VisualGen speeds up FLUX 3 Image Klein: 3.46x text-to-image and 4.45x image-to-image on one GPU, with cache and sparse attention costing LPIPS
Source illustration 20Original report ↗
Source illustration 21 — NVIDIA's TensorRT LLM VisualGen speeds up FLUX 3 Image Klein: 3.46x text-to-image and 4.45x image-to-image on one GPU, with cache and sparse attention costing LPIPS
Source illustration 21Original report ↗
Source illustration 22 — NVIDIA's TensorRT LLM VisualGen speeds up FLUX 3 Image Klein: 3.46x text-to-image and 4.45x image-to-image on one GPU, with cache and sparse attention costing LPIPS
Source illustration 22Original report ↗
Source illustration 23 — NVIDIA's TensorRT LLM VisualGen speeds up FLUX 3 Image Klein: 3.46x text-to-image and 4.45x image-to-image on one GPU, with cache and sparse attention costing LPIPS
Source illustration 23Original report ↗
Source illustration 24 — NVIDIA's TensorRT LLM VisualGen speeds up FLUX 3 Image Klein: 3.46x text-to-image and 4.45x image-to-image on one GPU, with cache and sparse attention costing LPIPS
Source illustration 24Original report ↗
Source illustration 25 — NVIDIA's TensorRT LLM VisualGen speeds up FLUX 3 Image Klein: 3.46x text-to-image and 4.45x image-to-image on one GPU, with cache and sparse attention costing LPIPS
Source illustration 25Original report ↗
Source illustration 26 — NVIDIA's TensorRT LLM VisualGen speeds up FLUX 3 Image Klein: 3.46x text-to-image and 4.45x image-to-image on one GPU, with cache and sparse attention costing LPIPS
Source illustration 26Original report ↗
Source illustration 27 — NVIDIA's TensorRT LLM VisualGen speeds up FLUX 3 Image Klein: 3.46x text-to-image and 4.45x image-to-image on one GPU, with cache and sparse attention costing LPIPS
Source illustration 27Original report ↗
Source illustration 28 — NVIDIA's TensorRT LLM VisualGen speeds up FLUX 3 Image Klein: 3.46x text-to-image and 4.45x image-to-image on one GPU, with cache and sparse attention costing LPIPS
Source illustration 28Original report ↗
Source illustration 29 — NVIDIA's TensorRT LLM VisualGen speeds up FLUX 3 Image Klein: 3.46x text-to-image and 4.45x image-to-image on one GPU, with cache and sparse attention costing LPIPS
Source illustration 29Original report ↗
Source illustration 30 — NVIDIA's TensorRT LLM VisualGen speeds up FLUX 3 Image Klein: 3.46x text-to-image and 4.45x image-to-image on one GPU, with cache and sparse attention costing LPIPS
Source illustration 30Original report ↗
Source illustration 31 — NVIDIA's TensorRT LLM VisualGen speeds up FLUX 3 Image Klein: 3.46x text-to-image and 4.45x image-to-image on one GPU, with cache and sparse attention costing LPIPS
Source illustration 31Original report ↗
Source illustration 32 — NVIDIA's TensorRT LLM VisualGen speeds up FLUX 3 Image Klein: 3.46x text-to-image and 4.45x image-to-image on one GPU, with cache and sparse attention costing LPIPS
Source illustration 32Original report ↗

Three minutes to your first working call

Create an account and an API key. Calls use your account balance. 30% off your first top-up. Top up ¥500, pay ¥350, receive ¥500 in credit. Top up ¥1000, pay ¥850, receive ¥1000 in credit You can still start with a ¥10 trial top-up.

Sources and editorial process

在 NVIDIA Blackwell GPU 上使用 FLUX 3 Image Klein 加速图像生成

AI assisted the analysis and translation of the cited sources, followed by automated quality checks. Recommendations are not test results. Consult the current catalog for availability and pricing.