P022 Training-run power signature
What are the iteration period, dip duty cycle and harmonic structure of a frontier training run's power trace at chip, rack and facility meter?
The power signature of a training run at chip, rack and facility meter.
Status
Started One normalized production training trace on DGX-H100 racks (no GPU count, model or absolute power): a 0.2-3 Hz band and an iteration dip recurring 'from once per second or less, to once every tens of seconds'. Dip duty cycle, step-time jitter and harmonic amplitudes at chip, rack and facility meter are not reported. Node level: Vercellino et al. (2026), also held, measure 4xH100 node power at 0.1 s for fine-tuning and training up to 16 nodes. Facility meter: simulation only (Ko & Zhu 2026, whose assumed 0.5-1.5 Hz iteration rate conflicts with Choukse; Vercellino's stitched facility profiles) and an unattributed once-a-second grid sag at a data-center substation (Mishra et al. 2025).
AI workload power traces like those in Figure 1 show FFT energy concentrated between 0.2–3 Hz (Figure 3)—close to known resonant modes of turbine-generator shafts and long transmission lines.
Posed by
- Ko & Zhu (2026), Wide-Area Power System Oscillations from Large-Scale AI Workloads (IEEE TPWRS):
there is an urgent need to improve the modeling of large-scale AI workload power profiles for grid-level studies.
(L183–185) (source states the problem but not the quantity the question asks for)
Also raised in: Canary Institute research.
Sub-questions
| ID | Sub-question | What the sources report |
|---|---|---|
| P022.1 | For a frontier training run, what are the iteration period (~1 s vs ~25 s), dip duty cycle and step-time jitter at GPU, rack and facility meter? | open |
| P022.2 | In a wall-side trace sampled >= 10 Hz during training, what are harmonic amplitudes above 0.5 Hz relative to the fundamental? | open |
| P022.3 | At given step-time jitter and dip duty cycle, what harmonic-summing gain (dB) over a single-bin detector is available? | open |
| P022.4 | Does the Kimpson-Baker-Graham minimum detectable training load differ from the power-side report's by a stated factor under matched conditions? | open |
Possibly relevant work found by automated search
Found by AI agents searching for this problem's sub-questions, mostly by keyword and citation in the OpenAlex scholarly database and on the web; a second AI pass reviewed each one, and we spot-checked some by hand. They seemed reasonably related, as far as one can tell from outside each paper's field. Some come from neighbouring fields, such as arms control and financial audit, and bear on the problem by analogy. Not reviewed against the status above; a paper listed here does not change that status. 928 of 1052 quotes checked word for word against the paper's text.
For P022.1: For a frontier training run, what are the iteration period (~1 s vs ~25 s), dip duty cycle and step-time jitter at GPU, rack and facility meter?
For P022.2: In a wall-side trace sampled >= 10 Hz during training, what are harmonic amplitudes above 0.5 Hz relative to the fundamental?
Data
- problems.csv — one row per problem
- problem_status.csv — status, scope and evidence per problem
- problem_sources.csv — the posed-by quote and other documents per problem
- leaves.csv — all sub-questions, including those without a checkable form
- leaf_sources.csv — every source row behind each sub-question