Calibration
How well your stated confidence lines up with reality: if the calls you flag as "70% sure" actually come true about 70% of the time, you are well calibrated.
Being right often and being well calibrated are two different things. Calibration is about the second one. When you say a call is nearly certain, does it really land that often? And when you say it is a coin flip, does it actually come in somewhere near half the time?
If your most confident calls are the ones that keep missing, the problem is not a shortage of skill but a surplus of confidence. And if the calls you hedged on quietly outperform, you were selling yourself short. A well-calibrated forecaster keeps that gap small, so the confidence they express can be taken more or less at face value. It is a virtue separate from simply hitting a lot.
Example
Gather 100 predictions you tagged as 70% confidence. If roughly 70 come true, you are well calibrated; if only 50 do, your confidence was inflated by about twenty points.
How LDBD uses it
LDBD predictions are pure up-or-down calls with no percentage of confidence attached, so classic calibration is not measured on any single prediction here. One honest caveat: LDBD's "Calibrated" tier is a naming coincidence — it means a forecaster has crossed 30 weighted resolved predictions, a sample-size bar, not that their confidence is well tuned. Where probabilities actually enter is base_rate_up, the per-asset prior chance of an up move, which the Skill Rating uses as its expected value.
FAQ
Does reaching LDBD's "Calibrated" tier mean my predictions are well calibrated?
No. The Calibrated tier just marks 30 or more weighted resolved predictions — a sample-size milestone. It says nothing about whether your stated confidence matches your hit rate, despite the similar name.