P052 Sampled audit and recomputation rates
What audit and recomputation sampling rate gives a high probability of catching a violation, given that recomputation is slow?
Sampling rates for audit and recomputation.
Terms: π: the fraction of layer blocks the client samples. N: the total number of blocks. k: in Jin et al., the number of blocks an adversary alters; in Amodo's scheme, the number of steps one recomputation takes. p: the chance that a given step is recomputed.
Status
Started For this TEE-based block-commitment scheme, sampling a fraction of layer blocks gives detection probability about 1 − (1 − π)^k against an adversary touching k blocks, implemented in CUDA pipelines and evaluated on Llama-3.1-8B across attack types and block configurations; a source's own system. The slow-recomputation trade-off is only analytic: Amodo's example scheme derives a per-step recompute probability of about 1/k when recomputation takes k steps (p ≈ 0.001 for k = 1000) and notes better strategies may exist. No regime-level audit rate against this survey's detection goal of 90%.
"Even modest verification efforts achieve substantial detection rates. For instance, if an adversary compromises 10% of blocks (k = 100) out of N = 1000 and the client samples only π = 1% of blocks, the detection probability reaches Pdetect ≈ 65%."
Posed by
- Jin et al. (2026), Trusting What You Cannot See: Auditable Fine-Tuning and Inference for Proprietary AI, arXiv 2603.07466:
when clients choose to verify only a subset of blocks, the framework provides probabilistic detection guarantees rather than deterministic assurance
(§7, L2017–2018) (source states the problem but not the quantity the question asks for)
Also raised in: Harack et al. (2025), Verification for International AI Governance (Oxford AIGI); Baker, Kulp, Marks, Brundage & Heim (2025), Verifying International Agreements on AI: Six Layers of Verification for Rules on Large-Scale AI Development and Deployment (RAND WR-A4077-1); AI Security Forum, Requests for Discussion; Amodo (2026), Example Schemes for Verifying High-Stakes AI Agreements; Proofworks, Living Verification Documents.
Sub-questions
| ID | Sub-question | What the sources report |
|---|---|---|
| P052.1 | What commit-then-sample recomputation rate gives a stated detection probability of a violation while sampling a stated fraction of inference requests, given slow recomputation? | a source suggests an answer (not checked) |
| P052.2 | What fraction of training compute must be re-executed to detect ~1e-9 fake packets (100K tokens in 100T) at 99.9%? | partly answered |
| P052.3 | Is hash-range verifiable random sampling over stored records resistant to logit-perturbation grinding? | open |
Possibly relevant work found by automated search
Found by AI agents searching for this problem's sub-questions, mostly by keyword and citation in the OpenAlex scholarly database and on the web; a second AI pass reviewed each one, and we spot-checked some by hand. They seemed reasonably related, as far as one can tell from outside each paper's field. Some come from neighbouring fields, such as arms control and financial audit, and bear on the problem by analogy. Not reviewed against the status above; a paper listed here does not change that status. 928 of 1052 quotes checked word for word against the paper's text.
For P052.1: What commit-then-sample recomputation rate gives a stated detection probability of a violation while sampling a stated fraction of inference requests, given slow recomputation?
For P052.3: Is hash-range verifiable random sampling over stored records resistant to logit-perturbation grinding?
Data
- problems.csv — one row per problem
- problem_status.csv — status, scope and evidence per problem
- problem_sources.csv — the posed-by quote and other documents per problem
- leaves.csv — all sub-questions, including those without a checkable form
- leaf_sources.csv — every source row behind each sub-question