Calibration at scale
Thousands of units sharing one calibration controller: availability, start-up time and logic-heat storms, from Part I's loop scaled up.
What this stage does
Every unit drifts like the module of chapter 6: its heaters wander, pilot tracking holds them, and when tracking loses a unit it stops and recalibrates. With thousands of units the question is whether one calibration controller can keep up. It can serve 4 illustrative units at a time by default; this chapter sweeps that parallelism P, the units U and the drift.
How it’s modelled
Two things change at scale. Tracking is shared: the controller dithers and reads pilots for P units at a time, so each unit gets P/U of the pilot updates it would get alone, which is Part I’s loop with its pilot period stretched ⌈U/P⌉ times. Logic heat adds 0.005 rad/√s illustrative of drift per heater. Running that loop gives each unit’s recalibration rate λ. Recalibrations queue: a failed unit waits for a free channel, then is out of service for 1 s (MEMS 0.75 s, non-volatile 50.5 s). The fleet is a machine-repair queue, solved exactly; a SimPy model adds fixed recalibration times and logic-heat storms.
It reduces to Part I. At U = 1 without logic heat, the queue model gives 0.6483 in service, and running the loop of chapter 6 directly at the same settings (0.1 rad/√s, 100,000 frames between pilots) gives an availability of 0.6483: the same number, as it must be.
What we found
Calibration scales if the controller does. With P = 4 the fleet keeps at least nine in ten units in service at every swept U up to 16,384 at the default drift (0.01 rad/√s illustrative), but falls below that from 4,096 units at three times the drift and from 1,024 at ten times. The collapse is sharp: once shared tracking is too thin, units fail faster than the controller can recalibrate them, and almost none are left in service. Raising P moves the cliff (from 256 units with P = 1 to 16,384 with the widest swept controller, P = 64, at the fastest drift).
Start-up takes time. Calibrating every unit once takes 4,096 s for 16,384 units with P = 4. Storms cost a tenth of the fleet. With hot logic, every busy/idle change (0.1 /s) knocks out a share of the units at once; at U = 64 and P = 1 the in-service fraction falls from 0.9821 to 0.8831.
Data table
| units U | P = 1 | P = 4 | P = 16 | P = 64 |
|---|---|---|---|---|
| 1 | 1 | 1 | 1 | 1 |
| 4 | 1 | 1 | 1 | 1 |
| 16 | 1 | 1 | 1 | 1 |
| 64 | 0.975 | 1 | 1 | 1 |
| 256 | 0.006284 | 0.9878 | 1 | 1 |
| 1,024 | 0.0007611 | 0.006284 | 0.9899 | 1 |
| 4,096 | 0.0001998 | 0.0007611 | 0.006284 | 0.99 |
| 16,384 | 4.757 × 10⁻⁵ | 0.0001998 | 0.0007611 | 0.006284 |
Data table
| units U | P = 1 | P = 4 | P = 16 | P = 64 |
|---|---|---|---|---|
| 1 | 1 | 1 | 1 | 1 |
| 4 | 1 | 1 | 1 | 1 |
| 16 | 1 | 1 | 1 | 1 |
| 64 | 1 | 1 | 1 | 1 |
| 256 | 1 | 1 | 1 | 1 |
| 1,024 | 0.02566 | 1 | 1 | 1 |
| 4,096 | 0.002873 | 0.02566 | 1 | 1 |
| 16,384 | 0.0005112 | 0.002873 | 0.02566 | 1 |
Data table
| units U | P = 1 | P = 4 | P = 16 | P = 64 |
|---|---|---|---|---|
| 1 | 1 | 1 | 1 | 1 |
| 4 | 4 | 4 | 4 | 4 |
| 16 | 16 | 16 | 16 | 16 |
| 64 | 62.4 | 64 | 64 | 64 |
| 256 | 1.609 | 252.9 | 256 | 256 |
| 1,024 | 0.7794 | 6.435 | 1,014 | 1,024 |
| 4,096 | 0.8182 | 3.118 | 25.74 | 4,055 |
| 16,384 | 0.7794 | 3.273 | 12.47 | 103 |
Data table
| units U | P = 1 | P = 4 | P = 16 | P = 64 |
|---|---|---|---|---|
| 1 | 1 | 1 | 1 | 1 |
| 4 | 4 | 1 | 1 | 1 |
| 16 | 16 | 4 | 1 | 1 |
| 64 | 64 | 16 | 4 | 1 |
| 256 | 256 | 64 | 16 | 4 |
| 1,024 | 1,024 | 256 | 64 | 16 |
| 4,096 | 4,096 | 1,024 | 256 | 64 |
| 16,384 | 16,380 | 4,096 | 1,024 | 256 |
What would need measured data
The drift rates, recalibration time and logic-heat figures are Part I’s illustrative values or extensions of them; the failure rates come from a finite run of the loop (zero where none were seen). What would pin the cliff down: measured drift of many units on one die, under a real workload’s heat, how a real controller shares its dither and pilot measurements, and whether recalibrating one unit disturbs its neighbours (the model assumes not).