P088 Proof-of-learning / proof-of-training-transcript
Is there a proof-of-learning/transcript scheme with a formal soundness bound against an adaptive spoofer, and at what verifier cost as a fraction of training?
Proof-of-learning / proof-of-training-transcript soundness and cost.
Terms: proof-of-learning (PoL) / proof-of-training-transcript (PoTT): schemes that produce evidence a specific training run actually happened as described, checkable by a verifier.
Status
Started Heuristic Proof-of-Training-Data defences (certified-randomness seed, local memorization tests) tested on GPT-2-124M and Pythia checkpoints against all attacks then published in the Proof-of-Learning literature, at about 1.3% of training compute for the verifier; language models only, single attacks only, composite and adaptive attacks and online RL untested, and the Verifier sees raw data and weights. No formal soundness bound exists: the earlier Proof-of-Learning family (Jia et al. 2021) was broken by Fang et al. (2023), a tested failure of that mechanism family; Shavit (2023) gives the formal PoTT definition and leaves provably secure methods to future work.
"Our methods can be run cheaply, adding as little as 1.3% of the original training run’s compute. … The verification strategies we describe are not provably robust, but are intended as an opening proposal"
Posed by
- Reuel, Bucknall et al. (2025), Open Problems in Technical AI Governance, compute sections:
finding efficient methods for proving the authenticity of training transcripts that scale to the largest models
(§5.2.2, p.29, L1553–1554) (source states the problem but not the quantity the question asks for)
Also raised in: Harack et al. (2025), Verification for International AI Governance (Oxford AIGI); Baker, Kulp, Marks, Brundage & Heim (2025), Verifying International Agreements on AI: Six Layers of Verification for Rules on Large-Scale AI Development and Deployment (RAND WR-A4077-1); Scher & Thiergart (2024), Mechanisms to Verify International Agreements About AI Development (MIRI); Aarne, Fist & Withers (2024), Secure, Governable Chips (CNAS); Shavit (2023), What Does It Take to Catch a Chinchilla? Verifying Rules on Large-Scale Neural Network Training via Compute Monitoring; Abdelghafar & Kulp (2026), Privacy-Preserving AI Verification via Minimal Information Disclosure; Ansari (2026), Hardware-Level Governance of AI Compute: A Feasibility Taxonomy for Regulatory Compliance and Treaty Verification; Kimpson, Baker & Graham (2026), Can Power Draw Constrain Covert Compute? Limits of Analogue Verification for AI Governance, arXiv 2610.07476; Peigné-Lefebvre, Nguyen & Wang (2026), Zero Knowledge Verification for Frontier AI Training Is Possible, arXiv 2606.05433.
Sub-questions
| ID | Sub-question | What the sources report |
|---|---|---|
| P088.1 | Is there a PoL/PoTT scheme with a formal soundness (uniqueness) bound against an adaptive spoofer with compute equal to the honest run? | open |
| P088.2 | For the best known spoofing attacks (incl. shard-containing models, synthetic gradient-inducing data), what is spoof cost as a multiple of honest training? | partly answered |
| P088.3 | What detection rate do code/data validation tests achieve against code or data designed to spoof partial re-execution (e.g. weight-overwrite code)? | open |
| P088.4 | Have PoL defences been tested against composite and adaptive attacks, on large models (a size threshold to be agreed) and non-text modalities? | partly answered |
| P088.5 | Do cheap all-segment tests detect 'glued' transcripts from different runs to a stated detection rate with a stated false-alarm budget? | partly answered |
| P088.6 | Is there a proof-of-training/commitment scheme for online RL / RL post-training where data is generated during training? | open |
| P088.7 | What is verifier cost as a fraction of training for the best PoL / spot-check protocol at a stated training-compute scale? | open |
| P088.8 | Is there a check costing a stated small share of training (a target to be agreed) (no re-execution, e.g. interactive queries) that bounds a run's FLOP from a weight shard? | open |
| P088.9 | Can a proof-of-training run with the verifier seeing only commitments plus sampled segments (no full code/data/weight disclosure)? | open |
| P088.10 | Has weight-snapshot hashing been used to verify a training-compute claim on a real large run? | partly answered |
| P088.11 | Is ZK-SNARK proving of distances between committed snapshots practical (proving time for billion-element vectors)? | a source suggests an answer (not checked) |
Sub-questions not yet stated precisely enough to check
| ID | Sub-question |
|---|---|
| P088.12 | Can training claims be verified without operator cooperation, and how is it confirmed that the declared work actually ran? |
Possibly relevant work found by automated search
Found by AI agents searching for this problem's sub-questions, mostly by keyword and citation in the OpenAlex scholarly database and on the web; a second AI pass reviewed each one, and we spot-checked some by hand. They seemed reasonably related, as far as one can tell from outside each paper's field. Some come from neighbouring fields, such as arms control and financial audit, and bear on the problem by analogy. Not reviewed against the status above; a paper listed here does not change that status. 928 of 1052 quotes checked word for word against the paper's text.
For P088.1: Is there a PoL/PoTT scheme with a formal soundness (uniqueness) bound against an adaptive spoofer with compute equal to the honest run?
For P088.2: For the best known spoofing attacks (incl. shard-containing models, synthetic gradient-inducing data), what is spoof cost as a multiple of honest training?
For P088.4: Have PoL defences been tested against composite and adaptive attacks, on large models (a size threshold to be agreed) and non-text modalities?
For P088.7: What is verifier cost as a fraction of training for the best PoL / spot-check protocol at a stated training-compute scale?
For P088.9: Can a proof-of-training run with the verifier seeing only commitments plus sampled segments (no full code/data/weight disclosure)?
For P088.11: Is ZK-SNARK proving of distances between committed snapshots practical (proving time for billion-element vectors)?
* Full text not reviewed; more likely to contain errors.
Data
- problems.csv — one row per problem
- problem_status.csv — status, scope and evidence per problem
- problem_sources.csv — the posed-by quote and other documents per problem
- leaves.csv — all sub-questions, including those without a checkable form
- leaf_sources.csv — every source row behind each sub-question