The accountability funnel
How many resolved predictions survive to a real verdict
(
confirmed/refuted).Resolved calls — the actual verdicts
Every graded prediction: what PIE called, the
confidence it assigned, and the open-source evidence that confirmed or
refuted it. Newest first.
Why predictions go ungraded
Most losses are evidence-matching failures
(
unresolvable / legacy expired) or ambiguous
partial signal — not wrong calls.Grading coverage by impact tier
Graded (green) vs ungraded (red), split by the stakes of the call.
Where the engine places its confidence
Histogram of predicted probability across all resolved forecasts.
Reliability diagram
Each point is a probability bucket: x = predicted, y = observed
frequency. Points should sit on the dashed line. Bubble size = sample count.
Where PIE is sharp — and where it isn’t
Calibration broken out by domain and impact tier: how often
each slice’s calls land (
hit rate), how honest the stated
probabilities are (Brier, lower = better), and whether the
engine over- or under-states its confidence in that slice.
Methodology: groups computed in-browser from the same
prediction_outcomes
payload; per-domain Brier prefers the resolver’s published
calibration_scores where available. Slices with fewer than 3 graded
calls are flagged low sample and make no confidence claim.
Grading rate by domain
Where forecasts go to die — volume vs. how often they close.
| Domain | Total | Graded | Rate |
|---|