P103 Training-data verification

Can a verifier confirm a declared dataset's provenance and composition produced a given model, and detect a tiny fraction of backdoor data?

Verifying training data provenance and composition.

Status

Started Proof-of-Training-Data tested on GPT-2 (124M) and Pythia (70M) checkpoints: catches gluing, interpolation, data subtraction and large data additions for about 1.3% of training compute; the verifier is given data, weights and code. The headline's backdoor clause is not met: the authors concede the method is not robust to small additions of harmful data (quoted by Reuel et al.). Composition rules (e.g. code-token fraction) and privacy-preserving membership proofs have designs only (Scher 2026; flexHEG Part 1); verifying that claimed data origins are uncompromised: no work.

We show experimentally that our verification procedures can catch a wide variety of attacks, including all known attacks from the Proof-of-Learning literature.
Choi, Shavit & Duvenaud (2023), Tools for Verifying Neural Models' Training Data, arXiv 2307.00682 (2023-07-02), L35–36

Posed by

Also raised in: Baker, Kulp, Marks, Brundage & Heim (2025), Verifying International Agreements on AI: Six Layers of Verification for Rules on Large-Scale AI Development and Deployment (RAND WR-A4077-1); Petrie, Aarne, Ammann & Dalrymple (2025), Overview of Flexible Hardware-Enabled Guarantees (flexHEG Part 1); Scher (2026), Verifying Restrictions on Frontier AI Research (MIRI); Harack (2026), AI Verification: Infrastructure for Prosperity, Governance, and Peace (Lawfare).

Sub-questions

IDSub-questionWhat the sources report
P103.1Does any proof-of-training-data protocol detect insertion of a small (stated-fraction) amount of backdoor data?partly answered
P103.2At frontier scale, can a verifier confirm a declared dataset produced a given model (commitments in a neutral DC) with a stated false-accept rate?partly answered
P103.3Is there a privacy-preserving protocol proving dataset membership to an auditor, with reported proof cost?open
P103.4Can a verifier check data-composition rules (e.g. code-token fraction to a stated precision, excluded data types, filtering) from transcript or weights?open

Sub-questions not yet stated precisely enough to check

IDSub-question
P103.5How can claimed data origins (public sets, third parties, synthetic generators) be verified as uncompromised?

Possibly relevant work found by automated search

Found by AI agents searching for this problem's sub-questions, mostly by keyword and citation in the OpenAlex scholarly database and on the web; a second AI pass reviewed each one, and we spot-checked some by hand. They seemed reasonably related, as far as one can tell from outside each paper's field. Some come from neighbouring fields, such as arms control and financial audit, and bear on the problem by analogy. Not reviewed against the status above; a paper listed here does not change that status. 928 of 1052 quotes checked word for word against the paper's text.

For P103.2: At frontier scale, can a verifier confirm a declared dataset produced a given model (commitments in a neutral DC) with a stated false-accept rate?

For P103.3: Is there a privacy-preserving protocol proving dataset membership to an auditor, with reported proof cost?

For P103.4: Can a verifier check data-composition rules (e.g. code-token fraction to a stated precision, excluded data types, filtering) from transcript or weights?

Data