Part I · Today's module · Chapter 10 of 18

Is it worth it? The break-even explorer

Residue width, baud, wavelengths, multiply/add cost and placement against the precomputed sweep, with the Amdahl ceiling.

The break-even explorerpaperdatasheetillustrative
sparse16, NTTs on the module, 59-bit residues, 14 GBd × 1 λ, r = 5, over the module's QSFP56 port, as modelled
0.0194×
slower than FPGA-only (51.5× the time)
Source: results.json → e9_bootstrap.logs.sparse16.speedup.optical. Amdahl ceiling for this trace (NTTs free): 1.858×.
303540455055600.010.1110residue width (bits)speed-up vs FPGA-only (×)break-evenAmdahl ceiling (NTTs free)
Trace
NTT placement
Link to the module
Baud (GBd)
Wavelengths
Multiply / add cost ratio r
Kernel level (experiment 7) at this width, baud × λ and r
FPGA butterfly units matched by the optical corenot computed (widths 8–55 bits; 14/56/112 GBd × 1/4 λ)
Amdahl bound for removing the 8 adds of a butterfly1.4×
Equal-area speed-up, w=17, recombine 0 add/plane1.167×
Equal-area speed-up, w=17, recombine 0.25 add/plane0.6829×
Equal-area speed-up, w=32, recombine 0 add/plane1.167×
Equal-area speed-up, w=32, recombine 0.25 add/plane0.5×
Every value shown is a computed point from the simulator’s results, or a straight line between two computed residue widths of the same sweep. The vendored results do not contain the full five-dimensional sweep, so most combinations say “not in the precomputed sweep” rather than guess; the buttons jump to points that were computed.
S=1(1−f)+f/sS = \frac{1}{(1 - \hl{amdahl}{f}) + \hl{amdahl}{f}/s}
S  ≤  11−fS \;\le\; \frac{1}{1 - \hl{amdahl}{f}}

What this stage does

This chapter asks the question plainly: when would sending NTTs through the module make an FHE bootstrap faster than an FPGA alone? The levers are the residue width (narrower primes mean fewer bit planes per butterfly but more limbs), the baud rate, the number of wavelengths, how expensive a modular multiply is relative to an add on the FPGA (r), and whether the NTTs go to the module entirely or are split with the FPGA.

How it’s modelled

Two experiments answer it. At the kernel level (experiment 7), a radix-4 butterfly on the FPGA costs (3 + 1)·r + 8 add-equivalents; the optical core removes the 8 adds but needs w symbol slots per butterfly and a shift-and-add recombination of w bit planes on the FPGA, at 0.25 adds illustrative per plane and output. At the bootstrap level (experiment 9) the recorded OpenFHE kernels are mapped onto FPGA, HBM, module and port for each design point. The explorer reads those precomputed results; it does not run either model.

Kernel level: the optical core's transform rate against FPGA butterfly unitspaperillustrative
102030405010⁸10⁹10¹⁰10¹¹width (bits)4-point transforms/s
Data table
Optical-core transform rate vs operand width
width (bits)14GBd x156GBd x1112GBd x1112GBd x4FPGA k=1FPGA k=16
81.698 × 10⁹6.79 × 10⁹1.358 × 10¹⁰5.432 × 10¹⁰4 × 10⁸6.4 × 10⁹
168.488 × 10⁸3.395 × 10⁹6.79 × 10⁹2.716 × 10¹⁰4 × 10⁸6.4 × 10⁹
177.988 × 10⁸3.195 × 10⁹6.391 × 10⁹2.556 × 10¹⁰4 × 10⁸6.4 × 10⁹
245.658 × 10⁸2.263 × 10⁹4.527 × 10⁹1.811 × 10¹⁰4 × 10⁸6.4 × 10⁹
324.244 × 10⁸1.698 × 10⁹3.395 × 10⁹1.358 × 10¹⁰4 × 10⁸6.4 × 10⁹
403.395 × 10⁸1.358 × 10⁹2.716 × 10⁹1.086 × 10¹⁰4 × 10⁸6.4 × 10⁹
482.829 × 10⁸1.132 × 10⁹2.263 × 10⁹9.053 × 10⁹4 × 10⁸6.4 × 10⁹
552.469 × 10⁸9.876 × 10⁸1.975 × 10⁹7.901 × 10⁹4 × 10⁸6.4 × 10⁹
Kernel level: equal-area speed-up against the multiply/add cost ratioillustrative
35790.511.5r = mul/add costspeed-up (×)break-even
Removing the 8 adds can at best give the Amdahl bound (dashed). Charging bit-plane recombination at a quarter of an add per plane pushes the hybrid below break-even.
Data table
Equal-area speed-up vs multiply/add cost
r = mul/add costAmdahl bound (8 adds removed)w=17, recombine 0w=17, recombine 0.25w=32, recombine 0w=32, recombine 0.25
31.6671.250.60611.250.4167
41.51.20.64861.20.4615
51.41.1670.68291.1670.5
61.3331.1430.71111.1430.5333
71.2861.1250.73471.1250.5625
81.251.1110.75471.1110.5882
91.2221.10.77191.10.6111
101.21.0910.78691.0910.6316

What we found

At the paper’s operating point the module does not pay. One wavelength at 14 GBd matches about 1.997 FPGA butterfly units on 17-bit operands. Removing the adds can give at most 1.667× (r = 3) to 1.2× (r = 10) per butterfly, and with recombination at its default cost the equal-area hybrid reaches 0.6829× of one butterfly unit; kernel break-even needs recombination at or below 0.05882 add/plane.

On a real bootstrap, below break-even everywhere the port allows. The best swept point with the module behind its QSFP56 port reaches 0.06774× (sparse16) and 0.09602× (full14), with the return port binding. Raising the baud does not help: the width sweep is identical at every baud because the port is already full.

What would have to change. All of these at once: much higher baud × wavelengths (the crossing appears at 112 GBd × 8 λ), narrow residues (28 bits), near-free recombination, a cheap FPGA multiply relative to an add (r = 3), and a port about 34.76 times wider. Then the speed-up is 1.098× (sparse16) and 1.064× (full14), against Amdahl ceilings of 1.858× and 1.402×: barely past break-even, because NTTs are only part of the bootstrap.

Energy points the same way: charging the module’s static power (9.7 W paper) to its throughput gives 35,779 pJ per transform against 56 pJ on the FPGA.

Energy per full-width 4-point transformpaperdatasheetillustrative
1020304050101001,00010⁴10⁵width (bits)energy (pJ)
Data table
Energy per full-width transform
width (bits)hybrid 14 GBd, k=1hybrid 112 GBd x4, no recombine costFPGA only
827,870314.956
1634,900581.856
1735,780615.156
2441,930848.756
3248,9701,11656
4056,0001,38256
4863,0301,64956
5569,1801,88356

What would need measured data

The break-even conditions rest on illustrative FPGA costs: the multiply/add ratio, the recombination cost per plane, the logic per butterfly unit and the energy per add. They also assume the module sustains its symbol rate behind a port with framing at 0.97 illustrative. Measured FPGA resource and power figures for the post-processing, the module’s sustained throughput through a real port, and its power split would move the numbers. A complete sweep over all five levers would need the simulator re-run and re-vendored; the explorer only shows the slices that were computed.