P112 Replay mismatches and dispute settlement

What is the baseline rate of benign replay mismatches, and can a dispute procedure attribute a mismatch to hardware fault versus cheating?

The rate of benign mismatches in replay checks, and whether a dispute procedure can tell a hardware fault from cheating.

Status

Started Delegation setting: when two executions of the same ML program disagree, a referee bisects by checkpoint hashes to one training step and one operator, re-executes it and rules; benign floating-point mismatches are removed by bitwise-reproducible operators, implemented and benchmarked on DistilBERT and Llama-3.1 1B and 8B in FP32. It finds which party is wrong, not whether the cause was a hardware fault or cheating; no baseline rate of benign replay mismatches is measured; Cankaya (2026) claims his own bitwise replay settles statistical false alarms, a self-report at the same stage; GPU-stack training replay at frontier scale not shown.

we design Verde, a dispute arbitration protocol that efficiently handles the large scale and graph-based computational model of modern ML programs. For (2), we build RepOps (Reproducible Operators), a library that eliminates hardware “nondeterminism” by controlling the order of floating point operations performed on all hardware. Our implementation shows that refereed delegation achieves both strong guarantees for clients and practical overheads for compute providers.
Arun et al. (2025), Verde: Verification via Refereed Delegation for Machine Learning Programs, arXiv 2502.19405 (2025-02-26), L47–53

Raised in: Cankaya (2026), A System Overview for Near-Term, Low-Trust AI Compute Verification (v0.2); Cankaya (2026), Catching Misreporting About ML Hardware Use by Turning Noise into Signal, Part II; Cankaya (2026), Compute Verification FAQs; Canary Institute research.

Sub-questions

IDSub-questionWhat the sources report
P112.1What is the baseline rate of benign replay mismatches (bit flips, software faults) per record?open
P112.2What is the false-alarm rate of statistical replay per record, and can bitwise replay settle such disputes?partly answered
P112.3What is each verification mechanism's measured false-positive rate before deployment?open
P112.4Can a written dispute procedure attribute a replay mismatch to hardware fault vs cheating without the prover reproducing it (e.g. on GPU stacks)?open

Possibly relevant work found by automated search

Found by AI agents searching for this problem's sub-questions, mostly by keyword and citation in the OpenAlex scholarly database and on the web; a second AI pass reviewed each one, and we spot-checked some by hand. They seemed reasonably related, as far as one can tell from outside each paper's field. Some come from neighbouring fields, such as arms control and financial audit, and bear on the problem by analogy. Not reviewed against the status above; a paper listed here does not change that status. 928 of 1052 quotes checked word for word against the paper's text.

For P112.1: What is the baseline rate of benign replay mismatches (bit flips, software faults) per record?

For P112.2: What is the false-alarm rate of statistical replay per record, and can bitwise replay settle such disputes?

Data