Data Literacy — How Much Should You Believe?

Use statistics to improve a decision, not decorate one.

Theory: start with the comparison

A stable has won five of its last twenty races. That's 25%. Whether it's impressive depends on the runners.

If those horses were expected to win eight races between them, five is disappointing. If they were expected to win two, it deserves a closer look. Neither comparison proves the future will look like the sample.

Before you trust a statistic, pin down the population, the period, the outcome and the baseline. "Good with sprinters" is too vague. "This trainer's runners over a defined sprint range, in these seasons, compared with their pre-race expected chances" is a question you can actually investigate.

Sample size is not a traffic light

There's no universal point at which a sample becomes fact. Twenty observations can reveal a large difference, while five hundred can leave a small one unclear.

It also matters what an observation is. Five hundred starts from a handful of closely related horses, one stable or one unusual season contain far less independent evidence than five hundred spread widely.

Ask how many races, horses and connections are behind the number. Check whether the result belongs mostly to one period, one runner or a few big-priced winners. A bigger sample only helps if it measures the right thing.

Put a range around the estimate

Five wins from twenty starts is a 25% win rate. A standard calculation (a 95% Wilson interval, which treats each start as an independent trial) puts the plausible range at roughly 11% to 47%. That's wide, and real races differ from each other more than the calculation assumes, so the true uncertainty is, if anything, larger.

Treat the range as a rough measure of how far the percentage could move with more data. It isn't a forecast for the next horse, whose task may be completely different.

The practical lesson is simple: the percentage on the screen is much less precise than its neat display suggests.

A win-rate range and an edge estimate are different calculations. Even a precisely measured win rate still has to be compared with the prices and the expected chances.

Hot streaks: unusual doesn't mean impossible

If five independent runners each have a fixed 20% chance, the probability that at least four of them win is about 0.67%. That's rare for one set of five picked in advance.

But a racing database contains many trainers, many time windows and many overlapping sets of five. Search them after the results are in and some will look extraordinary. The question changes from "could this stable produce a streak?" to "how often would a search this broad find one somewhere?"

A streak can also reflect better horses, easier races or a real change in preparation. Treat it as an open question: possibly noise, possibly a stable that has changed something. Find out what changed, then test whether knowing it improves the next decision.

Confounding: another explanation travels with the signal

Say Trainer A has a higher overall win rate than Trainer B, but A runs far more short-priced favourites. Comparing the raw percentages mixes the trainer's work with the quality and placing of the horses.

Or say a jockey seems to do well near their own lowest historical riding weight. It sounds like extra determination. But riding weight also reflects the weight the horse was set to carry, the race conditions and any allowance. The variable carries information you didn't mean to measure.

An internal MWP study looked at exactly this minimum-weight idea. A promising first pattern weakened a lot once runners were compared at similar carried weights, and other tests didn't support the proposed explanation. That's a useful result: an appealing story turned into a narrower, less confident claim.

"Controlling for" something isn't a magic phrase. Ask whether the comparison actually separates the proposed explanation from the alternative.

Many searches create many apparent discoveries

You can split racing by country, trainer, trip, age, draw, going and dozens of other fields. Keep splitting and some combinations will show spectacular returns by chance alone.

A good research habit: write the hypothesis and the test before you look at the result. When an idea comes out of exploring, label it exploratory and check it on later data you haven't touched.

For predictions that will be made in time, test in time order. Randomly shuffling a horse's past and future starts can hide problems that show up the moment you use the method for real. Every input must have been available when the decision would have been made.

The future must stay out of the inputs

A historical test of trainer form can't use a trainer profile calculated today if that profile includes later winners. A "last five starts" feature can't include the race being predicted.

The same goes for odds. A strategy that bets when the market opens can't pick horses because they shortened later. You can measure that movement afterwards to judge your timing, but you didn't have it when the market opened.

Historical teaching displays need dates too. A current table can be a useful reference, as long as it isn't presented as evidence a student could have used before an old race.

Be careful with the market benchmark

Scaling the implied odds so the field adds up to 100% gives you the market's estimate of the chances. In a deep market it's a very good estimate. It's still an estimate.

Here is the trap. In every complete win race the scaled chances add up to one, and there is exactly one winner. Add them up over a season of 1,000 races and the expected winners come to 1,000, which is also the number of actual winners. So the overall actual/expected ratio is 1.00 by construction, however good or bad the prices were. It tells you nothing about a model, or about whether particular price groups are priced correctly.

To test whether a method predicts well, compare it with other methods on the same held-out races. To test a betting approach, use prices you could actually have got and include the costs. Report the uncertainty in both.

Practical: a tempting trainer panel

Quiz: claims worth testing

Up next: the draw, the route and the difference between a real disadvantage and an underpriced one.