P104 Training-procedure and code verification
Can a declared architecture, optimiser schedule and code be checked against an executed run, catching a deliberately mis-declared one?
Checking training procedure and code against declarations.
Status
Started Delegation setting: a client fixes the program and several untrusted providers run it; a referee localises any divergence to one training step and one operator, re-executes it and rules, so an execution that departs from the declared program is caught given bitwise reproducibility (the RepOps library). Implemented and benchmarked on DistilBERT and Llama-3.1 1B and 8B training and inference, FP32 only, about 60% overhead on an A100. Not shown: a prover who both writes and mis-declares the code, frontier scale, or any check of the declaration's content (architecture, optimiser schedule) beyond re-execution; Choi et al. (2023) re-execute segments of a declared transcript on GPT-2 in the same spirit, for data rather than code.
Our implementation shows that refereed delegation achieves both strong guarantees for clients and practical overheads for compute providers.
Posed by
- Reuel, Bucknall et al. (2025), Open Problems in Technical AI Governance, compute sections:
Furthermore, verifying properties such as a system’s architecture or training procedure remain open questions.
(§5.3.1, p.29, L1591–1592)
Also raised in: Harack et al. (2025), Verification for International AI Governance (Oxford AIGI); Baker, Kulp, Marks, Brundage & Heim (2025), Verifying International Agreements on AI: Six Layers of Verification for Rules on Large-Scale AI Development and Deployment (RAND WR-A4077-1); Petrie, Aarne, Ammann & Dalrymple (2025), Overview of Flexible Hardware-Enabled Guarantees (flexHEG Part 1); Petrie & Aarne (2025), Technical Options for Flexible Hardware-Enabled Guarantees (flexHEG Part 2); Aarne & Petrie (2025), International Security Applications of Flexible Hardware-Enabled Guarantees (flexHEG Part 3); Scher (2026), Verifying Restrictions on Frontier AI Research (MIRI); Dean, Verification Plan (AI 2040); Proofworks, Living Verification Documents.
Sub-questions
| ID | Sub-question | What the sources report |
|---|---|---|
| P104.1 | Can a declared architecture, optimiser/hyperparameter schedule and data commitment be checked against an executed run, catching a deliberately mis-declared one? | open |
| P104.2 | Does a published suite exist for checking a training transcript against a declared plan, with catch rates for planted deviations, and what can partial (less sensitive) plans verify? | open |
| P104.3 | Has an automated checker detected planted capability-hiding modifications or novel methods in training code, and confirmed submitted code is what ran? | open |
| P104.4 | Can a compute-graph classifier label post-training type (RL vs supervised, RL amount) on held-out jobs? | open |
Sub-questions not yet stated precisely enough to check
| ID | Sub-question |
|---|---|
| P104.5 | Can two code versions (factored for validation, fast for accounting) be proven equivalent, and which validation methods (frequency analysis, inspection, formal) scale? |
| P104.6 | How much do imprecise scaling-law trends weaken consistency-with-trend checks? |
Possibly relevant work found by automated search
Found by AI agents searching for this problem's sub-questions, mostly by keyword and citation in the OpenAlex scholarly database and on the web; a second AI pass reviewed each one, and we spot-checked some by hand. They seemed reasonably related, as far as one can tell from outside each paper's field. Some come from neighbouring fields, such as arms control and financial audit, and bear on the problem by analogy. Not reviewed against the status above; a paper listed here does not change that status. 928 of 1052 quotes checked word for word against the paper's text.
For P104.1: Can a declared architecture, optimiser/hyperparameter schedule and data commitment be checked against an executed run, catching a deliberately mis-declared one?
For P104.2: Does a published suite exist for checking a training transcript against a declared plan, with catch rates for planted deviations, and what can partial (less sensitive) plans verify?
For P104.5: Can two code versions (factored for validation, fast for accounting) be proven equivalent, and which validation methods (frequency analysis, inspection, formal) scale?
Data
- problems.csv — one row per problem
- problem_status.csv — status, scope and evidence per problem
- problem_sources.csv — the posed-by quote and other documents per problem
- leaves.csv — all sub-questions, including those without a checkable form
- leaf_sources.csv — every source row behind each sub-question