P103 Training-data verification
Can a verifier confirm a declared dataset's provenance and composition produced a given model, and detect a tiny fraction of backdoor data?
Verifying training data provenance and composition.
Status
Started Proof-of-Training-Data tested on GPT-2 (124M) and Pythia (70M) checkpoints: catches gluing, interpolation, data subtraction and large data additions for about 1.3% of training compute; the verifier is given data, weights and code. The headline's backdoor clause is not met: the authors concede the method is not robust to small additions of harmful data (quoted by Reuel et al.). Composition rules (e.g. code-token fraction) and privacy-preserving membership proofs have designs only (Scher 2026; flexHEG Part 1); verifying that claimed data origins are uncompromised: no work.
We show experimentally that our verification procedures can catch a wide variety of attacks, including all known attacks from the Proof-of-Learning literature.
Posed by
- Reuel, Bucknall et al. (2025), Open Problems in Technical AI Governance, compute sections:
the authors concede that their approach is not robust to all potential attacks, in particular additions of small amounts of harmful data
(§5.1.1, p.27, L1443–1445)
Also raised in: Baker, Kulp, Marks, Brundage & Heim (2025), Verifying International Agreements on AI: Six Layers of Verification for Rules on Large-Scale AI Development and Deployment (RAND WR-A4077-1); Petrie, Aarne, Ammann & Dalrymple (2025), Overview of Flexible Hardware-Enabled Guarantees (flexHEG Part 1); Scher (2026), Verifying Restrictions on Frontier AI Research (MIRI); Harack (2026), AI Verification: Infrastructure for Prosperity, Governance, and Peace (Lawfare).
Sub-questions
| ID | Sub-question | What the sources report |
|---|---|---|
| P103.1 | Does any proof-of-training-data protocol detect insertion of a small (stated-fraction) amount of backdoor data? | partly answered |
| P103.2 | At frontier scale, can a verifier confirm a declared dataset produced a given model (commitments in a neutral DC) with a stated false-accept rate? | partly answered |
| P103.3 | Is there a privacy-preserving protocol proving dataset membership to an auditor, with reported proof cost? | open |
| P103.4 | Can a verifier check data-composition rules (e.g. code-token fraction to a stated precision, excluded data types, filtering) from transcript or weights? | open |
Sub-questions not yet stated precisely enough to check
| ID | Sub-question |
|---|---|
| P103.5 | How can claimed data origins (public sets, third parties, synthetic generators) be verified as uncompromised? |
Possibly relevant work found by automated search
Found by AI agents searching for this problem's sub-questions, mostly by keyword and citation in the OpenAlex scholarly database and on the web; a second AI pass reviewed each one, and we spot-checked some by hand. They seemed reasonably related, as far as one can tell from outside each paper's field. Some come from neighbouring fields, such as arms control and financial audit, and bear on the problem by analogy. Not reviewed against the status above; a paper listed here does not change that status. 928 of 1052 quotes checked word for word against the paper's text.
For P103.2: At frontier scale, can a verifier confirm a declared dataset produced a given model (commitments in a neutral DC) with a stated false-accept rate?
For P103.3: Is there a privacy-preserving protocol proving dataset membership to an auditor, with reported proof cost?
For P103.4: Can a verifier check data-composition rules (e.g. code-token fraction to a stated precision, excluded data types, filtering) from transcript or weights?
Data
- problems.csv — one row per problem
- problem_status.csv — status, scope and evidence per problem
- problem_sources.csv — the posed-by quote and other documents per problem
- leaves.csv — all sub-questions, including those without a checkable form
- leaf_sources.csv — every source row behind each sub-question