P114 End-to-end pilots, demonstrations and red-team exercises

What have published pilots and red-team exercises of the verification stack (including two-state experiments) detected, leaked, and how fast did they warn?

Running and publishing prototypes, pilots and red-team events of the verification stack, including two-state experiments.

Status

Started One 8xH100 server with prover and verifier on the same box; open-weight 7B-120B models; inference only; developer-run attacks (model substitution, LoRA append, append-on-trigger); 37,500 prompt runs; completeness ignored; a fixed threshold gave honest-prover failures and short-output false negatives. Not an independent red team, no national-security team, no two-state experiment, no leak or warning-time measurement; Reynolds & Foster's line-rate hashing benchmark is a second self-run component test. Red-team exercises of the stack and a joint international experiment: no work (Baker et al., early 2025: the listed challenges 'mostly receive little to no work').

Our results add further evidence that Token DiFR can identify different incorrect-model attacks reliably across a small range of prompts.
Amodo (2026), An Inference Verification Prototype — Stage 1 (Amodo Design note) (2026-06-29), L235

Posed by

Also raised in: Harack et al. (2025), Verification for International AI Governance (Oxford AIGI); Baker, Kulp, Marks, Brundage & Heim (2025), Verifying International Agreements on AI: Six Layers of Verification for Rules on Large-Scale AI Development and Deployment (RAND WR-A4077-1); Aarne, Fist & Withers (2024), Secure, Governable Chips (CNAS); Cankaya (2026), All Hands on Deck to Build the Datacenter Lie Detector; Cankaya (2026), Research Note: The Fundamentals and Feasibility of Secure Network Taps for Verifying AI Datacenter Use; Ansari (2026), Hardware-Level Governance of AI Compute: A Feasibility Taxonomy for Regulatory Compliance and Treaty Verification; Daley (2026), Proving the Negative: Three Warnings from Arms Control for the Age of AI (War on the Rocks); Wasil, Reed, Miller & Barnett (2024), Verification Methods for International AI Agreements; AI 2040 (2026), Get Involved in Verification; Amodo (2026), AI 2040 Plan A — How You Can Help; Amodo (2026), AI 2040 Plan A — Verification SITREP; Amodo (2026), An Inference Verification Prototype — Stage 1.

Sub-questions

IDSub-questionWhat the sources report
P114.1In a red-team exercise (including national-security teams) on monitored hardware, what fraction of undeclared FLOP ran and had results exfiltrated without detection, and for which hidden-run or hidden-datacenter strategies?open
P114.2Has a public bug bounty for AI-accelerator hardware security features or an open verification stack been run, with count of valid findings?open
P114.3Is cloud-provider compute reporting accurate under adversarial conditions: what fraction of deliberately structured runs does it flag in a red-team exercise?open
P114.4Has a two-state joint AI-compute verification experiment been run?open
P114.5In joint testing by agencies, labs, chip makers and hyperscalers, what do candidate systems detect, what do they leak, and how fast do they warn?open
P114.6Has a sub-scale demonstration of network taps plus a secure verification cluster, built from off-the-shelf parts and representative of treaty-grade needs, been built and publicly reported?open
P114.7Has an inference-only tap retrofit been run on a live serving cluster?open
P114.8In the existing demo, has request capture been moved out of the serving backend into a verifier-owned proxy or tap?open

Sub-questions not yet stated precisely enough to check

IDSub-question
P114.9Has an initial international verification design (including proof-of-work / proof-of-memory alternatives) been fully scoped, prototyped and red-teamed?
P114.10Should verification mechanisms rely on transparency or obscurity, and can doubt be red-teamed without publishing an evasion manual?
P114.11What implementation details does a company-auditing regime need, and does it survive red-teaming?

Possibly relevant work found by automated search

Found by AI agents searching for this problem's sub-questions, mostly by keyword and citation in the OpenAlex scholarly database and on the web; a second AI pass reviewed each one, and we spot-checked some by hand. They seemed reasonably related, as far as one can tell from outside each paper's field. Some come from neighbouring fields, such as arms control and financial audit, and bear on the problem by analogy. Not reviewed against the status above; a paper listed here does not change that status. 928 of 1052 quotes checked word for word against the paper's text.

For P114.1: In a red-team exercise (including national-security teams) on monitored hardware, what fraction of undeclared FLOP ran and had results exfiltrated without detection, and for which hidden-run or hidden-datacenter strategies?

For P114.2: Has a public bug bounty for AI-accelerator hardware security features or an open verification stack been run, with count of valid findings?

For P114.3: Is cloud-provider compute reporting accurate under adversarial conditions: what fraction of deliberately structured runs does it flag in a red-team exercise?

For P114.4: Has a two-state joint AI-compute verification experiment been run?

For P114.5: In joint testing by agencies, labs, chip makers and hyperscalers, what do candidate systems detect, what do they leak, and how fast do they warn?

* Full text not reviewed; more likely to contain errors.

Data