P077 Cross-hardware bit-exact replay and emulation
Can a computation be replayed or emulated bit-exactly on hardware other than the prover's (other SKUs, vendors, CPU), and at what slowdown?
Replaying or emulating computations bit-exactly on hardware other than the prover's (other SKUs, vendors, CPU emulation, verification-chip variants).
Terms: GEMM (general matrix multiply): the dense matrix-multiplication that dominates AI compute. MoE (mixture of experts): a model design that routes each input through only some of its sub-networks.
Status
Started Software-only CPU emulation of NVIDIA GPU inference arithmetic (L40, L40S, A100, H100 profiles), dense transformer blocks of Qwen3-4B plus one full forward pass; bit-exact against CUTLASS, and against cuBLAS through a per-SKU dispatch catalogue except nvjet kernels on H100 at short sequences. Not mixture-of-experts, not AMD, TPU or Ascend, not training. Slowdown clause: not measured; the appendix estimates CPU need and says the figures 'likely need to be multiplied by at least 10x' for reduction-tree emulation, so that clause is paper-only. Cross-hardware next-token agreement without emulation is about 85% one GPU generation apart (Hausenloy & Li 2026).
The emulator matches CUTLASS bit-exactly at every tested hardware/sequence-length combination.
Posed by
- Cankaya (2026), Catching Misreporting About ML Hardware Use by Turning Noise into Signal, Part II:
bitwise replay precision requires identical hardware. In international treaty verification, the Prover likely controls the supply chain for their accelerator type.
(intro, p.1; lines 24–26) (source states the problem but not the quantity the question asks for)
Also raised in: Baker, Kulp, Marks, Brundage & Heim (2025), Verifying International Agreements on AI: Six Layers of Verification for Rules on Large-Scale AI Development and Deployment (RAND WR-A4077-1); Cankaya (2025), Catching Misreporting About ML Hardware Use by Turning Noise into Signal; Cankaya (2026), A System Overview for Near-Term, Low-Trust AI Compute Verification (v0.2); Cankaya (2026), Bit-Exact AI Inference Verification Without Performance Tradeoffs, arXiv 2606.00279v2; Canary Institute research; Hausenloy & Li (2026), The Compute Verification Post; Karvonen et al. (2025), DiFR: Inference Verification Despite Nondeterminism; Rinberg et al. (2025), Verifying LLM Inference to Detect Model Weight Exfiltration; Peigné-Lefebvre, Nguyen & Wang (2026), Zero Knowledge Verification for Frontier AI Training Is Possible, arXiv 2606.05433; Amodo (2026), Example Schemes for Verifying High-Stakes AI Agreements; Proofworks, Living Verification Documents.
Sub-questions
| ID | Sub-question | What the sources report |
|---|---|---|
| P077.1 | Can a TEE-hardened 'verification chip' variant be made bit-identical in matmul output to the original SKU? | open |
| P077.2 | Can bitwise replay be done without prover-supplied hardware (e.g. CPU emulation of dense NVIDIA kernels)? | partly answered |
| P077.3 | How many test vectors confirm that a precompile spec matches H100 BF16 GEMM bit-for-bit, given accumulation order, rounding and subnormal handling differ per GPU? | open |
| P077.4 | What is the relative L2 difference between one training step's weight update on A100 vs B200 for a fixed model and batch? | open |
| P077.5 | What is next-token agreement / honest noise floor between hardware generations and vendors (NVIDIA vs Ascend, GPU vs TPU vs NPU), and is it below the shift from a substituted model? | open |
| P077.6 | In a pooled mixed-hardware honest null, does statistical replay still detect small changes (e.g. 4-bit quantisation) — and how large is the ~2% branching-token mismatch cross-vendor? | a source suggests an answer (not checked) |
| P077.7 | Has bit-exact emulation been shown for a full MoE forward pass with router? | open |
| P077.8 | Has bit-exact emulation (or bitwise reproducibility) been shown on AMD MI300 or Google TPU? | open |
| P077.9 | Can proprietary kernel dispatch (nvjet, cuBLAS) be recovered to 0 diffs, and does a per-SKU catalogue survive library minor versions? | partly answered |
| P077.10 | Has bit-exact emulation been shown for SSM/Mamba scans and for FlashAttention-3? | implied by a source, not stated as open |
| P077.11 | What wall-clock slowdown does bit-exact CPU/cross-vendor emulation impose per forward pass (vs ~10,000x for full chip emulation)? | partly answered |
| P077.12 | Does re-execution of sampled GPU kernels on independent hardware detect deliberately altered instruction semantics? | open |
Possibly relevant work found by automated search
Found by AI agents searching for this problem's sub-questions, mostly by keyword and citation in the OpenAlex scholarly database and on the web; a second AI pass reviewed each one, and we spot-checked some by hand. They seemed reasonably related, as far as one can tell from outside each paper's field. Some come from neighbouring fields, such as arms control and financial audit, and bear on the problem by analogy. Not reviewed against the status above; a paper listed here does not change that status. 928 of 1052 quotes checked word for word against the paper's text.
For P077.1: Can a TEE-hardened 'verification chip' variant be made bit-identical in matmul output to the original SKU?
For P077.2: Can bitwise replay be done without prover-supplied hardware (e.g. CPU emulation of dense NVIDIA kernels)?
For P077.3: How many test vectors confirm that a precompile spec matches H100 BF16 GEMM bit-for-bit, given accumulation order, rounding and subnormal handling differ per GPU?
For P077.4: What is the relative L2 difference between one training step's weight update on A100 vs B200 for a fixed model and batch?
For P077.5: What is next-token agreement / honest noise floor between hardware generations and vendors (NVIDIA vs Ascend, GPU vs TPU vs NPU), and is it below the shift from a substituted model?
For P077.6: In a pooled mixed-hardware honest null, does statistical replay still detect small changes (e.g. 4-bit quantisation) — and how large is the ~2% branching-token mismatch cross-vendor?
For P077.10: Has bit-exact emulation been shown for SSM/Mamba scans and for FlashAttention-3?
For P077.12: Does re-execution of sampled GPU kernels on independent hardware detect deliberately altered instruction semantics?
* Full text not reviewed; more likely to contain errors.
Data
- problems.csv — one row per problem
- problem_status.csv — status, scope and evidence per problem
- problem_sources.csv — the posed-by quote and other documents per problem
- leaves.csv — all sub-questions, including those without a checkable form
- leaf_sources.csv — every source row behind each sub-question