PondralMax drift, last completed run: 0.9 pp
Pondral runs the same frozen evaluation suite monthly against every production engine. We compare each run to the one before it and publish the per-engine drift. The number above is the largest engine-level change between the two most recent runs, which were 30 days apart.
Cadence changed 2026-08-03. This suite ran weekly until then and monthly since, so the runs listed here are spaced differently before and after that date, and the drift above was measured across 30 days rather than a full month. No run has been skipped: the next one is due on the first of the month.
Per-engine drift, last run vs prior
| Engine | Last run mean | Prior run mean | Delta | Status |
|---|---|---|---|---|
| ChatGPT | 77.7 | 78.2 | 0.5 pp | Stable |
| Claude | 74.9 | 75.9 | 0.9 pp | Stable |
| Gemini | 74.3 | 73.7 | 0.7 pp | Stable |
| Perplexity | 76.3 | 76.0 | 0.3 pp | Stable |
| Grok | 74.4 | 74.5 | 0.0 pp | Stable |
A delta under 5 percentage points is within normal run-to-run variance. 5 to 7.5 pp is flagged as moderate. Anything 7.5 pp or higher is flagged as drift and surfaced in the next methodology changelog entry with an explanation.
Recent runs
| Run date | Overall mean | ChatGPT | Claude | Gemini | Perplexity | Grok |
|---|---|---|---|---|---|---|
| Jul 12, 2026 | 75.0 | 75.0 | — | 75.0 | 75.0 | 75.0 |
| Jul 19, 2026 | 74.7 | 77.2 | 75.0 | 72.3 | 74.2 | 74.9 |
| Jul 26, 2026 | 75.4 | 78.0 | 75.0 | 73.6 | 75.0 | 75.5 |
| Aug 2, 2026 | 75.6 | 78.2 | 75.9 | 73.7 | 76.0 | 74.5 |
| Sep 1, 2026 | 75.6 | 77.7 | 74.9 | 74.3 | 76.3 | 74.4 |
How this works
Every month a cron job runs a fixed, frozen set of eval queries against every production engine. Each (query, engine) response is scored on a deterministic 4-factor subset of the rubric. Context is excluded from this eval because it uses an LLM judge whose own non-determinism would pollute the reproducibility signal. The full 5-factor rubric is the one customer scores use.
For the full rubric, weights, and rater configuration, see the methodology page. Every methodology change is logged in the changelog.
Methodology v2.5.0 · cohort v0-2026-05 · last run Sep 1, 2026 (2 weeks ago)