Yao Qizhi, Yang Zhilin and others propose TCSExam: 100 theoretical computer science tasks to evaluate AI agents; Kimi K3's graph construction exceeds reference by one edge
Leiphone reported on November 12, 2026 that researchers including Yao Qizhi and Yang Zhilin jointly proposed TCSExam, which evaluates AI agents on 100 theoretical computer science research tasks across four tracks: theorem proving, Solver, mathematical construction, and open conjectures. The report says Kimi K3, on a construction task with 124 points where triangles and quadrilaterals are forbidden, produced a graph with 630 edges, exceeding the 629-edge reference record and passing program verification; the report did not mention the relevant models' weights, licenses, or billing.

What changed
A Leiphone report dated November 12, 2026 says that Turing Award winner Yao Qizhi, dean of Tsinghua University's Institute for Interdisciplinary Information Sciences, and Yang Zhilin, founder of Moonshot AI, together with other researchers, proposed an evaluation framework called TCSExam. It divides 100 theoretical computer science research tasks into four tracks: 20 theorem-proving tasks requiring Lean 4 proofs; 30 Solver engineering tasks requiring reproducible algorithm programs; 20 mathematical construction tasks testing whether models can generate objects such as graphs and error-correcting codes that satisfy strict constraints; and 30 open conjectures allowing progress either by proving them or by giving counterexamples. The report gives the paper number as arXiv 2611.07962v1. The report says that on the construction task of connecting 124 points without allowing triangles or quadrilaterals, Kimi K3 produced a graph with 630 edges, exceeding the previous 629-edge reference record and passing program verification. The same report says GPT-6 Astra completed 14 of 20 research-level theorems and advanced the known proof range, under specific conditions, of a matrix conjecture over the five-element finite field from 8 dimensions to 9 dimensions; the proof was organized into a project with 71 Lean modules, recompiled in an independent environment, and checked for axiom dependencies. The report also says that in one attempt Kimi K3 completed 7 intermediate checkpoints with 87.5% coverage but did not complete the final derivation required for the main theorem; under the three guidance conditions Bare, Ladder, and Guided, Kimi K3 completed at least 3, 4, and 7 distinct theorems respectively, while DeepSeek V4.1 Flash 0910 completed 0, 2, and 6. The report mentions that some accepted counterproofs of Ryser and MIRUP came from retrieving and adapting OpenAI's publicly available results, and that the remaining 28 open conjectures did not receive complete solutions. All of the above are the research team's statements as relayed by the report.
What it means for developers
For developers, the transferable part of this news is not that one edge but the evaluation methodology itself. The report shows that the same model's number of completed theorems changes noticeably depending on whether intermediate lemmas are provided or proof ideas are given; this is only the experimental result reported, but it points to a general problem: any evaluation aimed at "autonomous completion," if intermediate steps are written into the prompt, is measuring something else, and when choosing a model for integration it is easy to conflate the two capabilities. TCSExam's approach is to check proofs with the Lean 4 kernel and audit dependencies, check algorithm outputs with rerunnable programs, isolate local derivations with checkpoints, and explicitly state that when proof A borrows from B and proof B depends on A, local checks passing separately do not constitute a complete proof. This layered approach can be carried directly into your own agent acceptance process, provided outputs can be mechanically checked, for example whether code compiles, whether constraints are satisfied, and whether a checker can recompute the same objective value. The report also mentions that some accepted Ryser and MIRUP counterproofs came from retrieving and adapting public results, which is a reminder that self-built task sets must guard against contamination from "answers already online." In addition, the report says the Solver track introduced standardized scoring relative to multiple runs of Kimi K3, meaning that within this framework Kimi K3 also serves as a reference point; when reading statements like "better than the reference," one should first confirm who the comparison object is. A practical constraint is that these tasks are research-level, few in number, and high in per-task time and cost, so treating them as daily regression tests is unrealistic; the report also does not say through what interface these models were invoked or how they were billed, so a single record should not be taken directly as a basis for procurement or migration.
Integration and evaluation
The following are conditional suggestions based on the report; they are inferences and operational recommendations, not verified facts. First, treat "how much scaffolding is in the prompt" as an explicit variable to test. The observable change in the report is that Kimi K3 completed at least 3, 4, and 7 distinct theorems under the Bare, Ladder, and Guided conditions respectively, while DeepSeek V4.1 Flash 0910 completed 0, 2, and 6. The choice to make is: if the workload requires the model to find key intermediate steps itself (long-horizon agents, open problems that cannot be decomposed in advance), use the no-intermediate-hint tier as the main reference; if the workflow already includes step decomposition, evaluate at the tier where ideas are provided. The precondition for this choice is whether the task can be reliably decomposed into intermediate goals in advance. A measurable criterion is to run the same batch of tasks once under each of the three guidance tiers and record the difference in the number of main theorems fully passed; if the tier with ideas provided is clearly higher than the no-hint tier, the model's autonomy in your scenario is limited, and budget should be reserved for manual decomposition or extra rounds. Second, replace acceptance by "having another model read it" with "artifact plus independent checker." The transferable mechanism in the report is the research certificate: theorems are checked by the Lean 4 kernel with dependency auditing, algorithms are submitted as rerunnable programs whose validity is re-judged and objective value recomputed by a checker, intermediate goals are checked separately under temporary assumptions, and the main theorem must be rechecked after removing unproven assumptions. The choice to make is: for tasks whose outputs can be mechanically verified, require the model to deliver both the result and a checking script, have your pipeline rerun it, and count success only if the rerun passes. The report gives a set of observable numbers using the IWLS 2023 logic synthesis task as an example, which is an example for that task rather than an overall conclusion for the Solver track: GPT-6 Astra generated valid circuits in all 100 instances, of which 22 beat the public reference circuits, 31 matched the reference level, and the other 47 used more AND gates; this shows that "valid" and "better" must be counted separately. The measurable criterion is to record the valid rate and the proportion better than the reference in two separate columns; otherwise, if only whether it runs is counted, a decline in optimization quality will not appear in the report. Third, count "near completion" separately. The typical case in the report is that in one attempt Kimi K3 completed 7 intermediate checkpoints with 87.5% coverage but did not complete the final derivation required for the main theorem, and the system retained the partial result but refused to recognize it as a complete proof. The choice to make is to split "partial completion" and "delivered and passed" into two fields rather than folding them into one total score; otherwise, capability changes on long tasks will be averaged away. The precondition is that you can decompose the task into independently checkable intermediate goals; the measurable criterion is to record both the main-theorem pass rate and the average checkpoint coverage, and when the two diverge, rely on the main-theorem pass rate.
Limits and open questions
Several uncertainties should be retained. This is an evaluation built by the researchers themselves: the report says that during development more than 40 candidate theorems were removed from the task set because they had already been solved during trial runs, indicating that difficulty calibration is still being adjusted. The 630-edge result is, according to the report, a program-verified construction; this article has not independently reproduced it, nor has it seen third-party review. The advance from 8 to 9 dimensions is a special case over the five-element finite field; the full Alon–Jaeger–Tarsi conjecture remains unsolved, and the report also says the remaining 28 open conjectures did not receive complete solutions, so a single record should not be read as general research capability. The numbers of theorems completed under the three guidance tiers cover only part of the theorem-proving track and cannot be treated as an overall ranking. The report does not mention the weights, licenses, or billing methods of Kimi K3 or DeepSeek V4.1 Flash 0910, nor does it give any timetable for model launch or open weights. As for the current KimiSeek catalog, Kimi K3 has an entry with the same name, but this is only the catalog's current state and cannot be used to infer the reproduction conditions, interface form, or any release plan of the experiments in the report; the version name DeepSeek V4.1 Flash 0910 that appears in the report has no exact corresponding entry in the current catalog, that version is not currently listed on KimiSeek, and the report does not say whether it is offered via API; related integration and cost information should be based on the vendor's statements and pricing page.
Illustrations from the original report



















