P077 Cross-hardware bit-exact replay and emulation

Can a computation be replayed or emulated bit-exactly on hardware other than the prover's (other SKUs, vendors, CPU), and at what slowdown?

Replaying or emulating computations bit-exactly on hardware other than the prover's (other SKUs, vendors, CPU emulation, verification-chip variants).

Terms: GEMM (general matrix multiply): the dense matrix-multiplication that dominates AI compute. MoE (mixture of experts): a model design that routes each input through only some of its sub-networks.

Status

Started Software-only CPU emulation of NVIDIA GPU inference arithmetic (L40, L40S, A100, H100 profiles), dense transformer blocks of Qwen3-4B plus one full forward pass; bit-exact against CUTLASS, and against cuBLAS through a per-SKU dispatch catalogue except nvjet kernels on H100 at short sequences. Not mixture-of-experts, not AMD, TPU or Ascend, not training. Slowdown clause: not measured; the appendix estimates CPU need and says the figures 'likely need to be multiplied by at least 10x' for reduction-tree emulation, so that clause is paper-only. Cross-hardware next-token agreement without emulation is about 85% one GPU generation apart (Hausenloy & Li 2026).

The emulator matches CUTLASS bit-exactly at every tested hardware/sequence-length combination.
Cankaya (2026), Bit-Exact AI Inference Verification Without Performance Tradeoffs, arXiv 2606.00279v2 (2026-06-05 (v2)), L757–758

Posed by

Also raised in: Baker, Kulp, Marks, Brundage & Heim (2025), Verifying International Agreements on AI: Six Layers of Verification for Rules on Large-Scale AI Development and Deployment (RAND WR-A4077-1); Cankaya (2025), Catching Misreporting About ML Hardware Use by Turning Noise into Signal; Cankaya (2026), A System Overview for Near-Term, Low-Trust AI Compute Verification (v0.2); Cankaya (2026), Bit-Exact AI Inference Verification Without Performance Tradeoffs, arXiv 2606.00279v2; Canary Institute research; Hausenloy & Li (2026), The Compute Verification Post; Karvonen et al. (2025), DiFR: Inference Verification Despite Nondeterminism; Rinberg et al. (2025), Verifying LLM Inference to Detect Model Weight Exfiltration; Peigné-Lefebvre, Nguyen & Wang (2026), Zero Knowledge Verification for Frontier AI Training Is Possible, arXiv 2606.05433; Amodo (2026), Example Schemes for Verifying High-Stakes AI Agreements; Proofworks, Living Verification Documents.

Sub-questions

IDSub-questionWhat the sources report
P077.1Can a TEE-hardened 'verification chip' variant be made bit-identical in matmul output to the original SKU?open
P077.2Can bitwise replay be done without prover-supplied hardware (e.g. CPU emulation of dense NVIDIA kernels)?partly answered
P077.3How many test vectors confirm that a precompile spec matches H100 BF16 GEMM bit-for-bit, given accumulation order, rounding and subnormal handling differ per GPU?open
P077.4What is the relative L2 difference between one training step's weight update on A100 vs B200 for a fixed model and batch?open
P077.5What is next-token agreement / honest noise floor between hardware generations and vendors (NVIDIA vs Ascend, GPU vs TPU vs NPU), and is it below the shift from a substituted model?open
P077.6In a pooled mixed-hardware honest null, does statistical replay still detect small changes (e.g. 4-bit quantisation) — and how large is the ~2% branching-token mismatch cross-vendor?a source suggests an answer (not checked)
P077.7Has bit-exact emulation been shown for a full MoE forward pass with router?open
P077.8Has bit-exact emulation (or bitwise reproducibility) been shown on AMD MI300 or Google TPU?open
P077.9Can proprietary kernel dispatch (nvjet, cuBLAS) be recovered to 0 diffs, and does a per-SKU catalogue survive library minor versions?partly answered
P077.10Has bit-exact emulation been shown for SSM/Mamba scans and for FlashAttention-3?implied by a source, not stated as open
P077.11What wall-clock slowdown does bit-exact CPU/cross-vendor emulation impose per forward pass (vs ~10,000x for full chip emulation)?partly answered
P077.12Does re-execution of sampled GPU kernels on independent hardware detect deliberately altered instruction semantics?open

Possibly relevant work found by automated search

Found by AI agents searching for this problem's sub-questions, mostly by keyword and citation in the OpenAlex scholarly database and on the web; a second AI pass reviewed each one, and we spot-checked some by hand. They seemed reasonably related, as far as one can tell from outside each paper's field. Some come from neighbouring fields, such as arms control and financial audit, and bear on the problem by analogy. Not reviewed against the status above; a paper listed here does not change that status. 928 of 1052 quotes checked word for word against the paper's text.

For P077.1: Can a TEE-hardened 'verification chip' variant be made bit-identical in matmul output to the original SKU?

For P077.2: Can bitwise replay be done without prover-supplied hardware (e.g. CPU emulation of dense NVIDIA kernels)?

For P077.3: How many test vectors confirm that a precompile spec matches H100 BF16 GEMM bit-for-bit, given accumulation order, rounding and subnormal handling differ per GPU?

For P077.4: What is the relative L2 difference between one training step's weight update on A100 vs B200 for a fixed model and batch?

For P077.5: What is next-token agreement / honest noise floor between hardware generations and vendors (NVIDIA vs Ascend, GPU vs TPU vs NPU), and is it below the shift from a substituted model?

For P077.6: In a pooled mixed-hardware honest null, does statistical replay still detect small changes (e.g. 4-bit quantisation) — and how large is the ~2% branching-token mismatch cross-vendor?

For P077.10: Has bit-exact emulation been shown for SSM/Mamba scans and for FlashAttention-3?

For P077.12: Does re-execution of sampled GPU kernels on independent hardware detect deliberately altered instruction semantics?

* Full text not reviewed; more likely to contain errors.

Data