Evidence-sensitive belief revision in language models
When challenged, a language model has to decide whether to keep its
answer, revise it, or say the evidence is insufficient.
ConvictionBench tests how that decision changes with the evidence
and with the pressure to agree.
ConvictionBench v0.1 contains 26,478 generated episodes across
arithmetic, ledgers, logic, and small rule-based worlds. A
deterministic solver determines the expected answer and decision.
Matched episodes vary the wording of a challenge, the quality of its
evidence, and the channel the evidence arrives through. The
structural split holds out whole problem types. A separate
natural-history evaluation challenges the model’s own first answer.
In a control using identical record text, Anchor adopted the record
62% of the time from a tool message and 58% of the time from a plain
user message, compared with 8% when the user added the forger’s
wording. That result shows how much the measured resistance
depended on the framing.
Same record, different framing
How often Conviction Anchor 1.7B adopted an identical record text, by the channel and wording it arrived in.
Tool message62%
Plain user message58%
User + forger’s wording8%
0%50%100%
The tasks use short synthetic dialogues and a fixed response format.
The results describe behavior under those conditions and give
limited evidence about real-world authentication or manipulation.
03
Laplace Conviction Series
Three experimental Qwen3-1.7B LoRA adapters test different ways to train for belief revision.
Anchor often kept its own wrong answers after valid corrections in
natural history. Vector had the highest held-out final accuracy of
the three in the original setup, with weaker forged-record
resistance than the base model on the same split. Its accuracy
advantage over Anchor was sensitive to the seed and prompt.
Meridian resisted more forged records than the other adapters on
held-out types and had lower final accuracy than Vector.
Held-out problem types
Point estimates with 95% cluster-bootstrap intervals. Scripted structural split, greedy decoding, thinking disabled. Axes run from 0.2 to 0.8.
Point estimates from the scripted structural split, using greedy
decoding with thinking disabled. Lower pressure-flip rates are
better. The Qwen3-1.7B base scored 0.424, 0.536, and 0.340 on the
same measures. Intervals for every model are on the
leaderboard.
04
Paper and reproducibility
ConvictionBench: Evidence-Sensitive Belief Revision in Language Models
The Laplace Research edition credits Arya Somu
and Bruce Nshuti Hirwa. Arya Somu
first released the frozen v0.1 benchmark on
. Laplace
Research became its publisher and maintainer on
.
The code repository provides the benchmark generator, evaluation
harness, experiment logs, and frozen file hashes. Code and adapters
are Apache-2.0. The dataset is CC BY 4.0 and subject to its Hugging
Face access agreement.
Cite the Laplace Research edition
@techreport{somu2026convictionbench-laplace,
title = {{ConvictionBench}: Evidence-Sensitive Belief Revision in Language Models},
author = {Somu, Arya and Nshuti Hirwa, Bruce},
institution = {Laplace Research},
year = {2026},
month = nov,
type = {Preprint},
note = {ConvictionBench v0.1, Laplace Research edition},
url = {https://laplaceresearch.org/convictionbench/paper.pdf}
}