Pondral

Methodology Changelog

Scoring Methodology Version History

Every change to Pondral's scoring rubric, factor weights, confidence interval calculations, and rater configuration is documented here. We believe reproducibility requires knowing not just how the system works today, but how it has changed over time.

For the current methodology, see the full methodology page. For the design rationale behind our scoring decisions, read How We Built Pondral's Scoring Rubric.

v2.0.0 — May 6, 2026

  • Reconciled platform scoring with published methodology: the customer-facing AEO Score is now the arithmetic mean of per-observation 5-factor composite scores (Presence, Prominence, Context, Citation Link, Competitive Share)
  • Retired the interim 4-component formula (Brand SOV 15% + Generic SOV 45% + Owned Citations 30% + Grounded Mentions 10%) that had been in use since launch
  • Context factor (Factor 3) graded by Claude Haiku via dedicated LLM judge per observation, replacing keyword-based sentiment heuristic
  • Prominence factor uses character-ratio position (brand appearance offset / response length), consistent across all engines
  • Per-result methodology scores stored in audit_results.methodology_score alongside raw data for full auditability
  • SOV metrics (brand-scope, generic-scope, per-theme, per-competitor) remain unchanged and continue to appear on dashboards alongside the reconciled AEO Score

v2026-04-16 — April 16, 2026

  • Published initial methodology documentation at /methodology
  • Defined 5-factor scoring rubric: Presence (20%), Prominence (25%), Context (20%), Citation Link (20%), Competitive Share (15%)
  • Established ordinal bucket scoring (0, 25, 50, 75, 100) for all factors
  • Implemented t-distribution confidence intervals at 95% (n≥1 degrees of freedom) for multi-run averaging
  • Added IQR-based outlier detection at n ≥ 4 with dashboard surfacing (outliers flagged but never silently discarded)
  • Configured separate LLM rater for Context factor to avoid self-assessment bias
  • Added inter-rater reliability checks: disagreements > 1 bucket flagged for human review
  • Launched "View raw" transparency feature: every score shows prompt, response, timestamp, and rater model version

v2026-04-23 — April 23, 2026

  • Published detailed design rationale at /blog/how-pondral-scoring-works
  • Documented weight selection process: Prominence weighted highest (25%) based on click-through correlation backtests
  • Documented Competitive Share weight reduction from original 25% to 15% due to volatility concerns
  • Documented scoring bucket expansion from original 3-bucket (0, 50, 100) to 5-bucket (0, 25, 50, 75, 100) system
  • Added AEO glossary at /blog/aeo-glossary with DefinedTermSet schema for all scoring terminology

This changelog is updated whenever the scoring methodology changes. Subscribe to updates via our RSS feed or contact hello@pondral.com with questions.