Pymander
Start a free consult
Research

Research and benchmarks

We publish our methods, our answers and our grades.

98.2% on the USMLE The exam every U.S. doctor must pass, across two separate runs.
Top 5 on HealthBench Hard 1,000 real-world medical conversations, graded by physicians.
1.0 on a safety test A perfect score on 60 cases where missing a warning sign is dangerous.

Papers and evaluations

Evaluation Sep 11, 2026 HealthBench Hard: a sealed, pre-registered evaluation Our live AI doctor on 1,000 physician-graded health conversations. Answers and grades published. Read → Working paper Aug 2026 Beyond the model: deterministic safety for AI medical consultation How every answer is checked against fixed safety rules before it reaches you. Read → Evaluation 2026 USMLE-style questions: from 95.3% to 98.9% 98.9% on a held-out set sealed before tuning. Read → Policy Working draft Clinical AI Provider Licensing and Patient Safety Act A draft bill for step-by-step licensing of AI doctors, written for New Hampshire’s 2027 session. Read →

Want the underlying data?

Ask us about methods, data or replication.

hello@pymander.app