accuracy
How often our prediction was right
Pokiq puts a price tag on every leak it finds: this is costing you so much per 100 hands. That is a prediction, and a prediction can be checked. This page keeps public score of how often we were right, including when the answer does not flatter us. As far as we know, no training product publishes its own misses; that is exactly why this page exists.
the score right now
…
The score is fetched live from the database, the same database that makes the predictions. There is never a cached number here.
What exactly gets counted
The leak finder spots leaks in your game and states what each one costs, in cents per 100 hands. The moment you fix a leak, that becomes a testable claim: this loss should now disappear. The measurement is whatever our own detectors find afterwards for the same player, in the same game format, over real hands.
A prediction only counts once it can be tested fairly. That means: the leak has been fixed, and at least 500 hands were played in that format before the fix and at least 500 after it. Why 500: the strictest of our own detectors wants to see 400 hands before it dares to report a leak at all, and 500 rounds that minimum up to five full blocks of the unit the prediction itself is stated in. Grading a prediction on fewer hands would be a measurement our own detector does not trust.
A hit means: at least half of the predicted saving demonstrably arrived. If the same leak comes back after the fix at half or more of its predicted cost, the prediction counts as a miss. The band is a factor of two because the prediction itself is a sample estimate; pretending it is accurate to the cent would claim a precision the measurement does not have. If the leak comes back more than once, the most expensive return counts, because that is the strictest reading against ourselves.
when we may show the number
The bar is not ours.
A percentage drawn from a handful of cases is not a measurement, it is noise with a percent sign. Published standards exist for when a proportion may be reported, and we adopted those instead of inventing a bar that happens to suit us.
At least thirty testable predictions
The reporting standard of the US National Center for Health Statistics (Data Presentation Standards for Proportions, Vital and Health Statistics series 2, number 175, 2017) forbids reporting a proportion from fewer than thirty cases. The same standard also forbids a percentage whose confidence interval is wider than 30 percentage points, or wider than 130 percent of the percentage itself. Only when every rule is met does a number appear here.
No cell smaller than eleven
The cell size suppression policy of the US Centers for Medicare and Medicaid Services forbids showing a cell of one through ten cases, and also any percentage from which such a cell can be computed back. For this page that means: both the number of hits and the number of misses must be at least eleven before the percentage appears. Our data does not fall under that policy; we adopt it because a borrowed bar with a citation beats an invented one.
Always with an interval
The percentage never comes alone: it carries a 95 percent Clopper-Pearson interval, the same construction the rest of our numbers use. That interval is deliberately the strict variant; choosing a narrower one would follow the standard in name and loosen it in practice.
Why there is no percentage here yet
Pokiq is young and the cohort is small. Below thirty testable predictions this page shows only the counter, and that is not a shortcoming but the promise itself: the number appears the moment the standard is met, hit or miss, and not a day earlier because it happens to look good. The counter above is the real score at this moment; what has to happen next is simply more fixed leaks with enough hands played after them.
The full definition above also lives as a comment on the database function this page calls, so the bar cannot quietly move without it showing in version history. How we handle verifiability in general is on how we build this.
next step
The prediction being scored here
The leak finder shows how such a price tag comes about: which leak was found, what it rests on, and what it costs you per 100 hands.
See the leak finder →