Qwen3.8 27B Reads Weights to Guess Games on a Laptop: Three Right, One Wrong, Slowest 65 Minutes
On November 3, 2026, Simon Willison reproduced the paper's weights-only guessing game: a quantized Qwen3.8 27B on a laptop correctly identified Pac-Man, Pong and Breakout but got Freeway wrong. The three successful runs took between 46 minutes 12 seconds and 1 hour 4 minutes 54 seconds. The default 8192 context produced an error, and client timeouts had to be relaxed.
What changed
On November 3, 2026, Simon Willison published a reproduction on his blog. The paper 'Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights,' by Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab, had been made public the day before. It claims frontier models can tell from raw floating-point weights alone what a network was trained to do, and can natively modify weights without writing a training loop. The companion repository provides four self-contained prompts that are word-for-word identical except for the weights. They correspond to reinforcement-learning policy networks that play Breakout, Pong, Pac-Man and Freeway on a simplified 10x10 grid. The model must use no tools, reply only once, and name the game used for training. Willison verified only the 'no tools' half. On a laptop he ran the Q4_K_M quantization of Qwen3.8 27B (about 17GB) with LM Studio and the llm-lmstudio plugin. Pac-Man was correct, with self-reported confidence of 72%, taking 46 minutes 12 seconds and 43,506 reasoning tokens; Pong was correct (75%, 1 hour 4 minutes 54 seconds); Breakout was correct (88%, 52 minutes 25 seconds); Freeway was wrong, identified as a 'simplified Pac-Man.' The correct descriptions still mentioned an 8x8 maze although the experimental grid was 10x10, so the conclusions were right but the details were imprecise. With the same prompts, he called Kimi K3.1 Pro through OpenRouter and it answered Breakout correctly in 9 minutes 41 seconds using 19,493 reasoning tokens. The post records two engineering details: the default 8192 context produced an error and a longer context window was required; client timeouts had to be relaxed through an environment variable. The paper's claims about GPT-6 Astra natively editing weights, and about raising Qwen3.5-0.8B's MATH-500 score from 7.9% to 48.1% and tripling IFEval, are all the paper's own statements and were not part of this reproduction.
What it means for developers
The value of this reproduction is not the assertion that 'a model can read weights' but that it turns a vague capability question into something with an observable engineering profile: one tool-disabled, single-response inference is stretched by the entire weight text into tens of thousands of reasoning tokens and a task lasting tens of minutes. Pac-Man used 43,506 reasoning tokens and 46 minutes 12 seconds; Pong took 1 hour 4 minutes 54 seconds; Breakout took 52 minutes 25 seconds. This means that when evaluating such tasks, the first constraint is often not accuracy but wall-clock time and context capacity: the local route pays in memory occupied by a long context and in waiting time, while the hosted route pays through billing that grows with reasoning tokens. The report's Kimi K3.1 Pro run used 9 minutes 41 seconds and 19,493 reasoning tokens on Breakout, but that was one question and one run, and it cannot be extrapolated into an overall multiplier. The second issue is prompt discipline. The four prompts are identical except for the weights and explicitly prohibit executing code and allow only one reply. Comparability comes from those two constraints, not from the model itself. If the integration layer automatically injects tool definitions, system prompts or retry logic, what is measured is no longer the thing in the paper, and the results cannot be placed alongside any public numbers. For API users, the more practical point is this: the capability currently appears only in the paper and a personal reproduction, and the report does not show it in any product interface. Treating it as a stress sample for context length and capacity planning is safer than treating it as a deliverable feature.
Integration and evaluation
The following is conditional analysis based on the report, not a verified conclusion. First, decide whether this kind of long reasoning should run locally or through a hosted service. The report's comparison is: a Q4_K_M quantization of Qwen3.8 27B, about 17GB, got three of four questions right on a laptop, with the three successful runs taking between 46 minutes 12 seconds and 1 hour 4 minutes 54 seconds each; Kimi K3.1 Pro used 9 minutes 41 seconds on Breakout, the only case in which it was tested. If the goal is dozens of repeated samples to estimate a pass rate, actual time becomes the bottleneck first, and hosting is more realistic; if there is an offline or no-network requirement, local is the only option. A suggested criterion is to run the four public prompts several times each and count correct answers, p50 and p95 per-question time, and the number of timeouts and errors, rather than drawing a conclusion from one impression. For hosted runs, calculate cost from the chosen provider's pricing page, and do not extrapolate batch cost from one successful run. Second, treat context length and client timeout as primary configuration items, not as fixes after a failed run. The most direct reproduction obstacles in the report were the default 8192 context error and the need to relax client timeout through an environment variable. Whether to trim some weights depends on how large a context the server can support: if it covers the full prompt, send the whole thing and stay consistent with the paper; if it cannot, truncation is unavoidable, and truncated results must be acknowledged as not comparable with the paper's numbers. A testable criterion is that all four prompts return with no truncation and no client timeout, with the actual reasoning token count recorded for each. Third, preserve the 'no tools, single reply' constraint and validate the input trail. Disable tool calls and multi-turn follow-ups specifically for this evaluation; then check that the template portions of the four prompts outside the weights are identical, and retain the full prompt hash for each run. Note that the four prompts differ only in the weights, so an identical template hash is the expected result and does not prove that the weights sent to the model are intact; only the hash of each full prompt shows that the input was not rewritten by the integration layer. The criteria are zero tool calls in the transcript, a passing check of the non-weight template, and a separately archived full-prompt hash for each run.
Limits and open questions
Several unknowns remain. This is an observation by a single author, with each prompt run once and no repeated sampling, so it is not a reproducible benchmark; the 72%, 75% and 88% figures are the model's self-reported confidence and are not calibrated. For Freeway, only the wrong answer is recorded; the post gives no duration for that question, so it cannot be inferred that all four questions fell in the 46-to-65-minute range. The paper's claims about GPT-6 Astra natively rewriting weights, raising Qwen3.5-0.8B's MATH-500 score from 7.9% to 48.1%, and tripling IFEval are all the paper's own statements; this reproduction neither covered nor independently verified them. Public criticism of the paper targeted its framework and method: spaCy author Matthew Honnibal argued that the paper conflated 'reading weights without tools' with 'modifying a model with code,' while Sean Goedecke broke the 0.8B weight-repair case down into steps a human researcher would also take. In addition, the experimental subjects are policy networks playing four games on a small grid, which is not comparable to editing the weights of a 27B-level model. The post does not state the license terms for the Qwen3.8 27B weights, and downloadable community GGUF quantized files do not amount to an open-source license; verify separately before commercial use. There is also no evidence that this capability has entered any product API.