P088 Proof-of-learning / proof-of-training-transcript

Is there a proof-of-learning/transcript scheme with a formal soundness bound against an adaptive spoofer, and at what verifier cost as a fraction of training?

Proof-of-learning / proof-of-training-transcript soundness and cost.

Terms: proof-of-learning (PoL) / proof-of-training-transcript (PoTT): schemes that produce evidence a specific training run actually happened as described, checkable by a verifier.

Status

Started Heuristic Proof-of-Training-Data defences (certified-randomness seed, local memorization tests) tested on GPT-2-124M and Pythia checkpoints against all attacks then published in the Proof-of-Learning literature, at about 1.3% of training compute for the verifier; language models only, single attacks only, composite and adaptive attacks and online RL untested, and the Verifier sees raw data and weights. No formal soundness bound exists: the earlier Proof-of-Learning family (Jia et al. 2021) was broken by Fang et al. (2023), a tested failure of that mechanism family; Shavit (2023) gives the formal PoTT definition and leaves provably secure methods to future work.

"Our methods can be run cheaply, adding as little as 1.3% of the original training run’s compute. … The verification strategies we describe are not provably robust, but are intended as an opening proposal"
Choi, Shavit & Duvenaud (2023), Tools for Verifying Neural Models' Training Data (Proof-of-Training-Data), sections 1 and 4 (2023-07-02), L71–76

Posed by

Also raised in: Harack et al. (2025), Verification for International AI Governance (Oxford AIGI); Baker, Kulp, Marks, Brundage & Heim (2025), Verifying International Agreements on AI: Six Layers of Verification for Rules on Large-Scale AI Development and Deployment (RAND WR-A4077-1); Scher & Thiergart (2024), Mechanisms to Verify International Agreements About AI Development (MIRI); Aarne, Fist & Withers (2024), Secure, Governable Chips (CNAS); Shavit (2023), What Does It Take to Catch a Chinchilla? Verifying Rules on Large-Scale Neural Network Training via Compute Monitoring; Abdelghafar & Kulp (2026), Privacy-Preserving AI Verification via Minimal Information Disclosure; Ansari (2026), Hardware-Level Governance of AI Compute: A Feasibility Taxonomy for Regulatory Compliance and Treaty Verification; Kimpson, Baker & Graham (2026), Can Power Draw Constrain Covert Compute? Limits of Analogue Verification for AI Governance, arXiv 2610.07476; Peigné-Lefebvre, Nguyen & Wang (2026), Zero Knowledge Verification for Frontier AI Training Is Possible, arXiv 2606.05433.

Sub-questions

IDSub-questionWhat the sources report
P088.1Is there a PoL/PoTT scheme with a formal soundness (uniqueness) bound against an adaptive spoofer with compute equal to the honest run?open
P088.2For the best known spoofing attacks (incl. shard-containing models, synthetic gradient-inducing data), what is spoof cost as a multiple of honest training?partly answered
P088.3What detection rate do code/data validation tests achieve against code or data designed to spoof partial re-execution (e.g. weight-overwrite code)?open
P088.4Have PoL defences been tested against composite and adaptive attacks, on large models (a size threshold to be agreed) and non-text modalities?partly answered
P088.5Do cheap all-segment tests detect 'glued' transcripts from different runs to a stated detection rate with a stated false-alarm budget?partly answered
P088.6Is there a proof-of-training/commitment scheme for online RL / RL post-training where data is generated during training?open
P088.7What is verifier cost as a fraction of training for the best PoL / spot-check protocol at a stated training-compute scale?open
P088.8Is there a check costing a stated small share of training (a target to be agreed) (no re-execution, e.g. interactive queries) that bounds a run's FLOP from a weight shard?open
P088.9Can a proof-of-training run with the verifier seeing only commitments plus sampled segments (no full code/data/weight disclosure)?open
P088.10Has weight-snapshot hashing been used to verify a training-compute claim on a real large run?partly answered
P088.11Is ZK-SNARK proving of distances between committed snapshots practical (proving time for billion-element vectors)?a source suggests an answer (not checked)

Sub-questions not yet stated precisely enough to check

IDSub-question
P088.12Can training claims be verified without operator cooperation, and how is it confirmed that the declared work actually ran?

Possibly relevant work found by automated search

Found by AI agents searching for this problem's sub-questions, mostly by keyword and citation in the OpenAlex scholarly database and on the web; a second AI pass reviewed each one, and we spot-checked some by hand. They seemed reasonably related, as far as one can tell from outside each paper's field. Some come from neighbouring fields, such as arms control and financial audit, and bear on the problem by analogy. Not reviewed against the status above; a paper listed here does not change that status. 928 of 1052 quotes checked word for word against the paper's text.

For P088.1: Is there a PoL/PoTT scheme with a formal soundness (uniqueness) bound against an adaptive spoofer with compute equal to the honest run?

For P088.2: For the best known spoofing attacks (incl. shard-containing models, synthetic gradient-inducing data), what is spoof cost as a multiple of honest training?

For P088.4: Have PoL defences been tested against composite and adaptive attacks, on large models (a size threshold to be agreed) and non-text modalities?

For P088.7: What is verifier cost as a fraction of training for the best PoL / spot-check protocol at a stated training-compute scale?

For P088.9: Can a proof-of-training run with the verifier seeing only commitments plus sampled segments (no full code/data/weight disclosure)?

For P088.11: Is ZK-SNARK proving of distances between committed snapshots practical (proving time for billion-element vectors)?

* Full text not reviewed; more likely to contain errors.

Data