P092 Confidentiality and information leakage of verification
Is there a verification design whose channel provably leaks only a declared set of fields, keeping user data and intellectual property private against a state adversary?
Information leakage from verification to the verifier, inspector code access, and minimal disclosure.
Terms: I(S; Z | Y): conditional mutual information, in bits: how much a released report Z reveals about a protected property S of the workload beyond what the authorised result Y already reveals.
Status
Started Leakage measured as estimated conditional mutual information on real physical-measurement datasets (GPU power, telemetry and electromagnetic traces, including H100 and B200 nodes) across six verification tasks, with one release certified by a Groth16 zk-SNARK; a source reporting its own method. Not a provable information-flow bound: estimation is unreliable for high-dimensional traces, protection holds only under the design distribution of the development data, and nothing is tested against repeated queries, data fusion or a state adversary (Daley: filterability against a state adversary is not established).
"Across these tasks, MID produces three releases with perfect held-out verification and zero measured collateral leakage, while the remaining tasks yield explicit privacy–utility frontiers."
Posed by
- Reuel, Bucknall et al. (2025), Open Problems in Technical AI Governance, compute sections:
How can TEEs and their applications be designed in a way that limits their potential for misuse
(§5.2, p.27, L1474 (Q53); see also §5.2.2 L1519–1520) (source states the problem but not the quantity the question asks for)
Also raised in: Baker, Kulp, Marks, Brundage & Heim (2025), Verifying International Agreements on AI: Six Layers of Verification for Rules on Large-Scale AI Development and Deployment (RAND WR-A4077-1); Scher & Thiergart (2024), Mechanisms to Verify International Agreements About AI Development (MIRI); Cankaya (2026), All Hands on Deck to Build the Datacenter Lie Detector; Cankaya (2026), Catching Misreporting About ML Hardware Use by Turning Noise into Signal, Part II; Abdelghafar & Kulp (2026), Privacy-Preserving AI Verification via Minimal Information Disclosure; Ansari (2026), Hardware-Level Governance of AI Compute: A Feasibility Taxonomy for Regulatory Compliance and Treaty Verification; Daley (2026), Proving the Negative: Three Warnings from Arms Control for the Age of AI (War on the Rocks); Wasil, Reed, Miller & Barnett (2024), Verification Methods for International AI Agreements; AI Security Forum, Requests for Discussion; Dean, Verification Plan (AI 2040); Proofworks, Living Verification Documents.
Sub-questions
| ID | Sub-question | What the sources report |
|---|---|---|
| P092.1 | Is there a verification design whose attestation channel provably reveals only a fixed, declared set of fields (formal or audited information-flow bound), so it cannot be repurposed for broad surveillance and keeps user data and IP private? | open |
| P092.2 | For a proposed one-bit or minimal compliance interface (Fixed Set alarm, offline-licence renewal, Auditor-in-a-Box verdict), what is its measured leakage I(S; Z | Y) beyond the authorised answer on a realistic workload mix, and which candidate report minimises it? | partly answered |
| P092.3 | Can mutual-information leakage be estimated reliably (not just as a lower bound) for high-dimensional traces and small datasets? | open |
| P092.4 | When the workload mix shifts (new model families absent from calibration), does a selected minimal-disclosure report keep I(S; Z | Y) below a stated bound? | open |
| P092.5 | Under repeated queries plus data fusion, how many bits beyond the compliance bit does a named attestation channel leak per month, and has filterability been shown at regime scale against a state adversary? | open |
| P092.6 | From daily aggregate FLOP attestations alone, can an analyst recover a lab's run start/end dates (experiment cadence) to a stated precision? | open |
| P092.7 | Can a verifier-grade off-chip power sensor (e.g. ~20 kHz) recover input tokens or weights from a datacenter GPU? | implied by a source, not stated as open |
| P092.8 | What is the upper bound on bits of prover secrets leaked to the verifier per replay session, including algorithmic insight from re-run software? | partly answered |
| P092.9 | Does lower-rate EM collection eliminate model-structure leakage to the verifier? | partly answered |
Sub-questions not yet stated precisely enough to check
| ID | Sub-question |
|---|---|
| P092.10 | What is the achievable privacy-versus-verification-strength trade-off for a given set of allowed controls and required utility? |
| P092.11 | How can analog or telemetry signals be processed for verification without leaking model information? |
| P092.12 | What stops a verifier with model access from testing sensitive model properties beyond the agreed claims? |
| P092.13 | How can engineers who inspect trusted-cluster data be prevented from exfiltrating private information through that channel? |
| P092.14 | How high is side-channel risk to the prover when the verifier cannot influence what code runs? |
| P092.15 | Can compute use be verified without low-level access (tokens, model weights, algorithms), and which minimal-access methods work? |
| P092.16 | How much code access do inspectors need to be confident they see the true code, given software can be modified or hidden quickly? |
| P092.17 | Can re-run software be constrained so the prover cannot sabotage verification through input parameters? |
| P092.18 | Can what inference/training happened be proven cryptographically without exposing weights or data, and is non-confidential sensitive-data storage enough? |
| P092.19 | Does a small-data-owner exemption make inference-side compute accounting imprecise? |
Possibly relevant work found by automated search
Found by AI agents searching for this problem's sub-questions, mostly by keyword and citation in the OpenAlex scholarly database and on the web; a second AI pass reviewed each one, and we spot-checked some by hand. They seemed reasonably related, as far as one can tell from outside each paper's field. Some come from neighbouring fields, such as arms control and financial audit, and bear on the problem by analogy. Not reviewed against the status above; a paper listed here does not change that status. 928 of 1052 quotes checked word for word against the paper's text.
For P092.1: Is there a verification design whose attestation channel provably reveals only a fixed, declared set of fields (formal or audited information-flow bound), so it cannot be repurposed for broad surveillance and keeps user data and IP private?
For P092.2: For a proposed one-bit or minimal compliance interface (Fixed Set alarm, offline-licence renewal, Auditor-in-a-Box verdict), what is its measured leakage I(S; Z | Y) beyond the authorised answer on a realistic workload mix, and which candidate report minimises it?
For P092.3: Can mutual-information leakage be estimated reliably (not just as a lower bound) for high-dimensional traces and small datasets?
For P092.6: From daily aggregate FLOP attestations alone, can an analyst recover a lab's run start/end dates (experiment cadence) to a stated precision?
For P092.7: Can a verifier-grade off-chip power sensor (e.g. ~20 kHz) recover input tokens or weights from a datacenter GPU?
For P092.13: How can engineers who inspect trusted-cluster data be prevented from exfiltrating private information through that channel?
Data
- problems.csv — one row per problem
- problem_status.csv — status, scope and evidence per problem
- problem_sources.csv — the posed-by quote and other documents per problem
- leaves.csv — all sub-questions, including those without a checkable form
- leaf_sources.csv — every source row behind each sub-question