P124 Validity of compute as the governed quantity
Does frontier capability still require large-scale training, and how fast does algorithmic progress make a fixed compute threshold obsolete?
Whether compute remains a valid proxy for capability.
Terms: N: model size. M: total communication between pods.
Status
Posed
Posed by
- Aarne & Petrie (2025), International Security Applications of Flexible Hardware-Enabled Guarantees (flexHEG Part 3):
algorithmic progress could eventually mean that even computations that fall below this initial FLOP threshold could be dangerous.
(App A, Ratchets, p52, L2373–2374)
Also raised in: Baker, Kulp, Marks, Brundage & Heim (2025), Verifying International Agreements on AI: Six Layers of Verification for Rules on Large-Scale AI Development and Deployment (RAND WR-A4077-1); Scher & Thiergart (2024), Mechanisms to Verify International Agreements About AI Development (MIRI); Shavit (2023), What Does It Take to Catch a Chinchilla? Verifying Rules on Large-Scale Neural Network Training via Compute Monitoring; Scher (2026), Verifying Restrictions on Frontier AI Research (MIRI); Proofworks, Living Verification Documents.
Sub-questions
| ID | Sub-question | What the sources report |
|---|---|---|
| P124.1 | Does frontier capability still require large-scale training: of models setting a new state of the art in a given year, what fraction were trained above a stated compute threshold (given distillation and new architectures)? | open |
| P124.2 | What fraction of consequential algorithmic innovations required more than a stated amount of experiment compute to validate? | open |
| P124.3 | For innovations shown to be compute-dependent, what minimum experiment scale (FLOP) was needed to see the large-scale benefit? | implied by a source, not stated as open |
| P124.4 | Below what model size N and total inter-pod communication M does a covert workload lose a named dangerous capability (i.e. which violations are significant)? | open |
Sub-questions not yet stated precisely enough to check
| ID | Sub-question |
|---|---|
| P124.5 | How fast does algorithmic progress make sub-threshold compute dangerous, i.e. how long can a fixed compute threshold prevent a given capability? |
| P124.6 | How many tokens per minute can be tolerated unmonitored? |
| P124.7 | Is small-scale deployment of frontier models, outside the regime's scope, dangerous on its own? |
| P124.8 | Which mechanisms are specific to the transformer / pre-training paradigm, and would they survive a paradigm change? |
Possibly relevant work found by automated search
Not searched: no technical result was expected for this problem.
Data
- problems.csv — one row per problem
- problem_status.csv — status, scope and evidence per problem
- problem_sources.csv — the posed-by quote and other documents per problem
- leaves.csv — all sub-questions, including those without a checkable form
- leaf_sources.csv — every source row behind each sub-question