P127 Cost and overhead
What total overhead (as % of training/facility cost) does the full verification stack impose in a pilot, and can it stay far below the monitored hardware's cost?
Total cost and overhead of verification.
Status
Started One mechanism measured in the lab, not a stack in a pilot: making a small Llama model's computation reproducible for dispute checking costs about 60% extra time for training and inference on one A100. Device-cost estimate: monitoring hardware of hundreds to a few thousand dollars per rack against racks costing millions (Cankaya 2026, an order-of-magnitude estimate that leaves out guarding, inspection, recomputation and staff). No full verification stack has been piloted, so a cost of about 1% of a facility's cost is not shown.
both inference and training on Llama-3.11B incur around 60% overhead RepOps on an A100 GPU
Posed by
- Cankaya (2026), Compute Verification FAQs:
with currently available mechanisms, the necessary measures would be heavy-handed and costly, in order to be reliably enforced.
(When will the mechanisms be ready…?; L74; live page, read 2026-10-21) (source states the problem but not the quantity the question asks for)
Also raised in: Shavit (2023), What Does It Take to Catch a Chinchilla? Verifying Rules on Large-Scale Neural Network Training via Compute Monitoring; Canary Institute research; Moon, Vedula, Geneson & Bar-on (2025), Strategies and Detection Gaps in a Game-Theoretic Model of Compute Governance (RAND RRA3686-1).
Sub-questions
| ID | Sub-question | What the sources report |
|---|---|---|
| P127.1 | What total overhead, as a share of training cost, does the full snapshot + transcript + verification stack impose in a pilot, against a stated cap (a target to be agreed) rather than 20%? | implied by a source, not stated as open |
| P127.2 | What is the full deployed cost (BOM + guarding + inspection) of tap-based verification per $ of monitored hardware, and can it stay orders of magnitude below that hardware under extreme security requirements? | partly answered |
| P127.3 | Is the 2.5%-of-facility verification budget target a one-time capex share or an annual share, per its original source? | open |
| P127.4 | What is the per-account monthly CSP cost of monitoring FLOP vs power vs cluster size vs data ingress? | open |
Possibly relevant work found by automated search
Not searched: no technical result was expected for this problem.
Data
- problems.csv — one row per problem
- problem_status.csv — status, scope and evidence per problem
- problem_sources.csv — the posed-by quote and other documents per problem
- leaves.csv — all sub-questions, including those without a checkable form
- leaf_sources.csv — every source row behind each sub-question