P078 Compute accounting: MFU, HFU and declared operations

Can a model predict achievable hardware utilisation closely enough to detect padded or undeclared-efficiency compute, at a tolerable false-alarm rate?

MFU/HFU modelling and detecting padded or undeclared-efficiency compute.

Terms: MFU / HFU (model-FLOP utilisation / hardware-FLOP utilisation): the fraction of a chip's peak compute a training run actually uses - two accounting conventions that differ in how they count recomputed work.

Status

Analyzed A proposed method with a desk analysis of its error sources: estimate achievable utilisation for a declared workload and cluster, compare with declared operations, and back it with physical signatures when an operator's efficiency exceeds the estimate. No model has been validated against measured runs, no padding or no-op detector has been built, and the false-alarm rate for honest operators is only flagged as a risk ('honest Provers [may be] unable to account for their estimated compute use'). Attacks named without defences shown: secret backend optimisation, speculative decoding (1.5-3x), Pareto-inefficient menu choices, restructuring to minimise logged operations. Cankaya's system overview proposes a resource-occupation simulation cross-compared with throughput, also paper-only.

For a given workload type and cluster, optimal MFU could be coarsely approximated as the MFU known to have been achieved on similar workloads and clusters, on which there is private and public data (Epoch AI, 2024b). Optimal MFU could be more precisely estimated with theoretical modeling (Erdil and Schneider-Joseph, 2024; Erdil, 2025). Still, significant error bars may remain.
Baker, Kulp, Marks, Brundage & Heim (2025), Verifying International Agreements on AI: Six Layers of Verification (RAND WR-A4077-1) (2025-07-25 (v2)), L3040–3045

Posed by

Also raised in: Baker, Kulp, Marks, Brundage & Heim (2025), Verifying International Agreements on AI: Six Layers of Verification for Rules on Large-Scale AI Development and Deployment (RAND WR-A4077-1); Scher & Thiergart (2024), Mechanisms to Verify International Agreements About AI Development (MIRI); Cankaya (2026), A System Overview for Near-Term, Low-Trust AI Compute Verification (v0.2); AI Security Forum, Requests for Discussion; Proofworks, Living Verification Documents.

Sub-questions

IDSub-questionWhat the sources report
P078.1Given a workload and cluster specs, does a model predict achievable MFU / resource occupation within a stated tolerance of measured runs, including latency-optimised legitimate uses?open
P078.2Under a conservative-MFU rule, what false-alarm rate results for honest operators?open
P078.3What is the measured HFU/MFU ratio (recomputation) for frontier training, and what throughput does forbidding recomputation cost?partly answered
P078.4Can a check detect padded or no-op computation that inflates apparent operations by a stated fraction (a target to be agreed) within a declared power envelope?open
P078.5Can a verifier detect undeclared efficiency gains (secret backend optimisation, speculative decoding, Pareto-inefficient menu choices, restructuring to minimise logged FLOP) that free hidden capacity?partly answered

Sub-questions not yet stated precisely enough to check

IDSub-question
P078.6Does counting declared model operations need new R&D?

Possibly relevant work found by automated search

Found by AI agents searching for this problem's sub-questions, mostly by keyword and citation in the OpenAlex scholarly database and on the web; a second AI pass reviewed each one, and we spot-checked some by hand. They seemed reasonably related, as far as one can tell from outside each paper's field. Some come from neighbouring fields, such as arms control and financial audit, and bear on the problem by analogy. Not reviewed against the status above; a paper listed here does not change that status. 928 of 1052 quotes checked word for word against the paper's text.

For P078.1: Given a workload and cluster specs, does a model predict achievable MFU / resource occupation within a stated tolerance of measured runs, including latency-optimised legitimate uses?

For P078.4: Can a check detect padded or no-op computation that inflates apparent operations by a stated fraction (a target to be agreed) within a declared power envelope?

Data