Machine Learning — Use the Baseline Properly

A consistent estimate is useful even when it needs questioning.

Theory: what the model contributes

A model can compare more historical situations than you can hold in your head, and it applies the same method to every horse on the card, including the ones you find boring. That makes it a good starting point.

MWP's Expected Rating estimates a horse's likely performance from its form and other inputs. Read it alongside its definition and the evidence behind the horse. Predicting a lightly raced horse is a different task from predicting an established runner in familiar conditions.

A model can be consistent and still wrong in a systematic way. Data errors, changing populations of horses and thin coverage all affect it. The useful questions are where its estimates hold up, where they don't, and whether your adjustment actually improves them.

A rating, a ranking and a probability are different outputs

An expected rating describes a level of performance. A ranking puts the runners in order. A win probability describes a horse's chance within the whole field.

They're related, and each answers its own question. A small expected-rating advantage over one rival is a different thing from the same advantage over fourteen, and the uncertainty around each horse matters too.

A probability model can turn ratings, the field and other inputs into chances that add up across the race. A model can also blend its own view with market information. MWP's Expected Rating is built from form and never sees the odds. These are separate modelling choices, so don't assume a displayed rating has already done all of them.

If the product shows only a rating, the betting probability is still yours to estimate. If it shows a probability, check the model's description and when its information was taken.

The top pick is not the bet

An invented favourite with a 45% chance at 2.20 has an expected net return of 0.45 × 2.20 − 1 = −1%. Another runner with a 14% chance at 11.00 has an expected net return of +54%.

The arithmetic is easy because the probabilities are given. In real races, getting them right is the hard part, and a disagreement between a ranking and the market can't do that work for you.

Soul Rush's case in Chapter 12 asks whether a well-rated horse could be overlooked in a field dominated by a famous rival. The rating makes him worth investigating. A gap between the form ranking and the market ranking is a reason to look; only the probabilities can show whether the price is wrong.

Know what the model already sees

Assume the model may already know what is printed on the card: trainer changes, the draw, layoffs and pace, for example.

Equally, a field existing in a database doesn't prove the model uses it well. A coarse surface category or an incomplete trip flag can miss something specific. That's a reason to investigate a limitation, not a licence to add a bonus every time you recognise a familiar factor.

A useful manual adjustment names what it adds. "I watched the race" isn't enough. "The last run was badly restricted, the input I'm using doesn't show that, and today's route may be different" is an explanation you can test.

Peaks and expectations

Expected ratings often sit below a horse's best recent figure, because a peak isn't the typical outcome. That can be right even for a horse in good form.

Every lower model estimate still needs checking. A young horse may be improving, the history may be thin, or the conditions may have changed. Compare the model's expectation with your evidence instead of picking whichever number flatters your selection.

Readiness is a good example. If the model already accounts for the time since the last run and the stage of the campaign, those facts aren't new information from you. A reliable preparation detail outside its inputs might be.

The body-weight and trials ideas from earlier chapters remain research candidates unless a product release explicitly says they are collected and used.

What honest evaluation looks like

Test a model on later races it wasn't trained on, using only information available at the moment it claims to predict. Check the data processing too: a trainer aggregate or a revised historical record can leak later information without the model doing anything obviously suspicious.

Compare against meaningful baselines on the same races. Beating random selection is a much weaker result than improving on a strong market-based estimate.

For probability forecasts, look at both the scores and the calibration. Calibration asks whether events given similar probabilities happen at about those rates over a large enough sample. A model that calls a lot of horses 20% shouldn't see only 10% of them win, race after race.

Calibration isn't everything. A forecast that gives every runner an average, uninformative chance can be well calibrated and still miss the important differences. Proper probability scores, such as a clearly stated version of the Brier score, compare forecasts on the same outcomes. Cash returns, measured as ROI at prices you could actually have taken, are a separate test that brings in prices and execution.

We don't use the win rate of the highest-rated horse as proof of how well the Expected Rating model performs. They're different tests.

Test the human layer too

Save the baseline before you change anything. Record the adjusted figure or probability and your reason. Later, compare the original and adjusted estimates on the same races, not just on the bets you remember.

Your adjustments may help in one kind of situation and hurt in another, and a short run of winners won't tell you which. Resist slicing the results into ever smaller successful categories after the fact.

One of the best uses of a tracker is finding out that a favourite personal angle adds confidence without improving your forecasts. It's uncomfortable information, and worth having.

Practical: model versus you

Quiz: model claims

Up next: turn a race opinion into a full price line, then decide whether anything is worth buying.