← All model news

Future Doctor MedGPT reported to beat 250 tertiary-hospital doctors 88 to 12; ablation points to workflow

According to a November 12, 2026 report by Xinzhiyuan, a multicenter study of 40 difficult lung cancer cases says Future Doctor MedGPT beat 250 Grade 3A tertiary-hospital doctors by a case-level score of 88 to 12, with unaided doctors averaging 53.26 points against MedGPT's 85.29; the ablation experiment described in the report says removing the workflow costs nearly 24 points, and the report does not say whether the weights or an API are open.

Published Updated

What changed

According to a November 12, 2026 report by Xinzhiyuan, Future Doctor worked with the thoracic surgery departments of domestic Grade 3A hospitals on a multicenter study that selected 40 difficult cases with unclear guideline recommendations out of 100 complex lung cancer cases as the exam paper. Before seeing the answers, 20 experts wrote six scoring rules for each case, 240 rules in total, of which 40 were safety red lines. The comparison involved 250 Grade 3A tertiary-hospital doctors from 98 medical institutions, more than 60% of them chief or associate chief physicians. The report says that without assistance the doctor group averaged 53.26 while Future Doctor MedGPT scored 85.29; in case-level one-to-one comparisons against 750 unaided doctor plans, MedGPT scored higher 660 times, and the score given in the report is 88 to 12, with its answers on average more than 40% shorter than the doctors'. The report also gives a model comparison under a different scoring standard: MedGPT 96.22 overall, while the GPT-5.4, Claude Opus 4.7 and Gemini 3.1 Pro listed in the report scored 71.38, 61.81 and 54.62 respectively. The same report says that in the ablation experiment, taking apart the entire workflow and letting only the underlying model answer directly costs nearly 24 points, while removing only the multidisciplinary consultation module costs 16 points; on the human-machine collaboration dimension, unaided doctors scored 53.40, those given only retrieval references scored 61.05, and those who reviewed the full strategy report scored 66.27.

What it means for developers

The part of this system that is worth studying for developers is not any single model but the workflow. The workflow described in the report has three stages: at ingestion, the literature is broken into evidence units that each record only one conclusion, extracted from the source text by a program, so the model can only cite and cannot invent; patient information is organized into a state table that updates as test results come in, with updates requiring a doctor's signature; then up to six specialty agents each give their own opinion and disagreements are kept verbatim, producing a standard path, a frontier path and verification tasks, which pass through virtual-patient simulation and adversarial review along five dimensions — safety, tail risk, evidence and dosage, temporal execution, and marginal value — where results may only be kept, repaired or discarded, and typically two to three rounds are needed before release. For teams building vertical-domain agents, these are transferable design constraints, not model capabilities you can call directly. It should be made clear that the report is about the Future Doctor MedGPT closed system; the report does not mention that it has open weights and gives no developer-facing API or licensing information, so for API users this news changes evaluation methods and architectural trade-offs rather than offering an immediately substitutable integration option. Another easily overlooked point is that the same report gives both a system score of 96.22 versus 71.38 — that is, with retrieval and multi-agent review — and acknowledges that removing the workflow costs about 24 points. Reading the two sets of numbers together supports a conditional judgment: that model comparison is closer to a comparison of a full workflow against bare models, and should not be treated as a ranking of model capability under the same scaffolding.

Integration and evaluation

The following suggestions are conditional analysis, not facts from the report. First, quantify the workflow before touching the underlying model. What changes in the report are the ablation numbers: taking apart the whole workflow and leaving only the underlying model to answer directly costs nearly 24 points, and removing only the multidisciplinary consultation module costs 16 points. The choice to make is to first turn the evidence retrieval layer, the multi-agent consultation and the adversarial review into switches that can be disabled one by one, and only then talk about swapping models. This suggestion holds only if you have stable, pre-written scoring rules; if the standard is still changing, the score differences you measure cannot be attributed to a module rather than to the scoring criteria. The measurable criterion is the score difference from toggling each module under the same rule set, while also recording answer length, because the report says the system's answers are more than 40% shorter than the doctors' — a sudden change in length often indicates that scores are being bought by padding with words. Second, the trade-off between a structured evidence base and conventional retrieval-augmented generation. The report says evidence is broken down at ingestion into units that each record a single conclusion, and can only be copied from the source text by a program, with the model restricted to citing it; for lung cancer alone, more than 13,000 papers were parsed, producing over 200,000 structured evidence units. What you have to decide is whether to invest labor in this kind of evidence layer first or to hold the line with conventional retrieval-augmented generation. Its preconditions are that hallucinations are costly, that the domain has verifiable text such as guidelines and drug labels, and that you have people who can check extraction quality; if the corpus is messy and the questions are wide open, doing retrieval first is more realistic. The measurable criteria are citation precision — whether an extracted citation really supports that conclusion — and the share of numbers in the output that cannot be traced back to a source. Third, evaluate system against system and model against model separately. What changes in the report is that two sets of numbers appear together: in the model comparison MedGPT scores 96.22 against the reported GPT-5.4 71.38, Claude Opus 4.7 61.81 and Gemini 3.1 Pro 54.62, while on the human-machine collaboration dimension the figures are 53.40, 61.05 and 66.27. The choice to make is to run at least two baselines on your own use-case set — the full system, and the underlying model with the scaffolding removed — rather than comparing someone else's system with retrieval and review directly against your own bare model. The precondition is that you can replicate the other side's workflow; if you cannot, use switched versions of your own system as the control. The measurable criterion is whether rankings stay stable when the scoring standard changes; if they flip, the conclusions are sensitive to scaffolding and cannot be used directly for model selection. Fourth, choose the form of human-in-the-loop clearly. The report says doctors given only retrieval references scored 61.05, and those who reviewed the full strategy report reached 66.27, and that updates to the patient state table only count with a doctor's signature. The choice is whether the product exposes only retrieval snippets, or outputs a full strategy report for review and keeps a signature confirmation step. That depends on whether the workflow already has an accountable confirmation step, and whether the report's length exceeds the reviewer's time budget. The measurable criteria are the score difference between the retrieval-only and full-report arms on the same set of cases, and the actual rate at which doctors modify the plans.

Limits and open questions

The scoring rules in the report were drawn up in advance by experts invited by the research team, the cases number only 40 and are concentrated in a single disease, lung cancer, and the report does not mention independent third-party replication, so these numbers should be treated as self-reported results under the researchers' own standard; the frontline expert comments cited in the report also come from the report itself and are supportive opinions. The model version names and scores that appear in the report cannot be verified outside that source. The human-machine comparison, the model comparison and the human-machine collaboration sets of numbers use different standards, as the report itself distinguishes, and cannot be converted into one another or ranked directly. Most importantly: Future Doctor MedGPT currently has no corresponding entry on KimiSeek, so it should not be treated as a model already listed on that platform; the report also does not say whether its weights are open, whether it offers a developer API, or what license it uses, and any information about integration and availability should be taken from the vendor's own statements.

Three minutes to your first working call

Create an account and an API key. Calls use your account balance. 30% off your first top-up. Top up ¥500, pay ¥350, receive ¥500 in credit. Top up ¥1000, pay ¥850, receive ¥1000 in credit You can still start with a ¥10 trial top-up.

Sources and editorial process

250名三甲医生输给中国AI!引发中美医疗权威热议

AI assisted the analysis and translation of the cited sources, followed by automated quality checks. Recommendations are not test results. Consult the current catalog for availability and pricing.