P127 Cost and overhead

What total overhead (as % of training/facility cost) does the full verification stack impose in a pilot, and can it stay far below the monitored hardware's cost?

Total cost and overhead of verification.

Status

Started One mechanism measured in the lab, not a stack in a pilot: making a small Llama model's computation reproducible for dispute checking costs about 60% extra time for training and inference on one A100. Device-cost estimate: monitoring hardware of hundreds to a few thousand dollars per rack against racks costing millions (Cankaya 2026, an order-of-magnitude estimate that leaves out guarding, inspection, recomputation and staff). No full verification stack has been piloted, so a cost of about 1% of a facility's cost is not shown.

both inference and training on Llama-3.11B incur around 60% overhead RepOps on an A100 GPU
Arun et al. (2025), Verde: Verification via Refereed Delegation for Machine Learning Programs, arXiv 2502.19405 (2025-02-26), L118

Posed by

Also raised in: Shavit (2023), What Does It Take to Catch a Chinchilla? Verifying Rules on Large-Scale Neural Network Training via Compute Monitoring; Canary Institute research; Moon, Vedula, Geneson & Bar-on (2025), Strategies and Detection Gaps in a Game-Theoretic Model of Compute Governance (RAND RRA3686-1).

Sub-questions

IDSub-questionWhat the sources report
P127.1What total overhead, as a share of training cost, does the full snapshot + transcript + verification stack impose in a pilot, against a stated cap (a target to be agreed) rather than 20%?implied by a source, not stated as open
P127.2What is the full deployed cost (BOM + guarding + inspection) of tap-based verification per $ of monitored hardware, and can it stay orders of magnitude below that hardware under extreme security requirements?partly answered
P127.3Is the 2.5%-of-facility verification budget target a one-time capex share or an annual share, per its original source?open
P127.4What is the per-account monthly CSP cost of monitoring FLOP vs power vs cluster size vs data ingress?open

Possibly relevant work found by automated search

Not searched: no technical result was expected for this problem.

Data