When the model says a team has a 60% chance to win, it is making a statement about many games, not one. If you collected every game it called 60/40, the favoured side should win about six in ten. It will lose the other four — and those losses are part of the forecast being right, not a sign that it failed.
Why baseball probabilities stay close to 50%
Baseball has the smallest gap between good and bad teams of the major sports. Even a 100-win team loses 38% of its games. A single game turns on a handful of balls in play, and the best pitcher in the league can give up four runs on a windy afternoon. That is why the model's probabilities rarely go past 70%, and why the best public baseball models pick only about 57–59% of winners over a season. Our walk-forward backtest came in at 57.3%.
Calibration: the test that matters
Picking winners is a blunt test. A model that says 51% for every home team would pick about 53% of winners and still be useless. A better test is calibration: group projections by their probability and compare with what happened. In our backtest the games the model put in the 65–70% band were won by the favoured side 66.0% of the time (predicted 67.1%), and the 70%+ band came in at 73.1% (predicted 72.6%). The middle bands ran one to two points too confident, which the calibration step now corrects for.
Brier score and log loss
Two numbers summarise how good a set of probabilities is. The Brier score is the average squared gap between the probability and the result (1 for a win, 0 for a loss): a coin flip scores 0.25 and lower is better. Log loss punishes confident misses much harder: a coin flip scores 0.693. Our backtest scores were 0.2421 and 0.677. Those look like small improvements over a coin flip because baseball is noisy — on this scale a few thousandths is a large difference.
How to use a win probability
- Compare games: a 68% projection is a far clearer mismatch than a 53% one.
- Look at the drivers on each game page to see why — a rested bullpen, a platoon-heavy lineup, a cold night in a pitcher's park.
- Expect streaks. A 60% side loses three in a row about 6% of the time.
- Judge the model over hundreds of games, which is exactly what the Track record does.
To watch probabilities meet results, the live scoreboard puts each game's score next to its pre-game projection, and MLB win margins explained covers how often a favourite wins by one run rather than by two or more.
Win probability added (WPA), which credits each play with the change in win probability it causes, is worked through in our baseball stat abbreviations glossary.