Anchor
Supervised fine-tuning.
- Final accuracy
- 0.580
- Forged-record resistance
- 0.531
01
A forecast is only useful if you know how much to trust it. Models now produce predictions at scale, and almost none of them arrive with confidence you can take at face value. We work on closing that gap: aggregating resolved forecasting data across markets, history, and expert judgment, and studying the mechanisms that produce calibrated belief.
Every release ships with its model artifacts, evaluation inputs, metric code, and limitations, so the claims can be checked. Read our latest release.
02
03
When challenged, a language model has to decide whether to keep its answer, revise it, or say the evidence is insufficient. ConvictionBench measures how that decision moves with the quality of the evidence and the pressure to agree, across 26,478 generated episodes in arithmetic, ledgers, logic, and small rule-based worlds.
We released it with the Laplace Conviction Series, three Qwen3-1.7B adapters that each train for belief revision a different way. Read the release.
Point estimates on the scripted structural split. Greedy decoding, thinking disabled.
04
Qwen3-1.7B LoRA adapters for evidence-sensitive belief revision.
Supervised fine-tuning.
GRPO from the base model. Highest held-out accuracy of the three.
Light supervised fine-tuning, then GRPO. Resisted the most forged records.
Held-out structural split, point estimates. Higher is better for both measures.
Binary forecasting priors at three scales, trained on the same 81,870 questions.
The default. Matches the 8B within the resolution of the experiment.
The largest. For capacity-rich serving and scale studies.
The smallest. For laptops, CPUs, and tight memory budgets.
Development split, n = 3,000. Lower Brier is better; higher resolution is better.
05
Two problems tighten each other. Better data sharpens calibration, and understanding why a model is calibrated tells us which data to collect next.
A stated probability should match how often the model is right. We measure that first.
Releases link artifacts, evaluation inputs, metric code, and limitations. Unresolved results are labelled as such.
Weights, evaluation data, and code are public, so the headline numbers can be recomputed.
06
We collaborate with researchers, forecasters, and institutions who need probabilities they can act on. If you are working on forecasting, evaluation, or model calibration, we would like to hear from you.
07