Task 4 of 5
Sampling chops up time. Storing the samples chops up amplitude:
with bits bits there are 2bits code values to cover the whole −1…+1
range, so the step between neighbouring codes is 2 / 2^bits and every sample gets
nudged to the nearest one.
step = 2 / Math.pow(2, bits);
q = Math.round(value / step) * step; // NOT Math.floor
error = q - value; // in −step/2 … +step/2
Math.round is load-bearing. Rounding down quantises just as
successfully, but every error then has the same sign, so instead of jitter centred on zero you
get a DC offset of −step/2 riding on the signal — and you measurably throw away a whole bit.
(Run this task with Math.floor if you want to watch it happen: every SNR drops by
about 6 dB.)
Which is the rule worth having. Signal-to-noise ratio is
10·log10(signal power / error power), and halving the step — one more bit —
quarters the error power, which is 10·log10(4) ≈ 6 dB. Do not take that on
faith: the loop below runs 4, 6, 8, 10 and 12 bits and you can read the slope straight off the
console.
The kernel writes two planes again — output: [256, 2], plane 0 the quantised
signal and plane 1 the error — so one pass gives you both the result and the thing you want to
measure. That is fusion, the same trick "Reductions" ends on: compute it where you already have
the value in hand rather than paying for a second pass.
signal to bits bits into plane 0
and the rounding error into plane 1, then log the SNR at each of the five depths.bits as a kernel argument and derive step = 2 / Math.pow(2, bits)this.thread.y === 0) holds Math.round(value / step) * stepquantised - valueconsole.log the SNR in dB for each of 4, 6, 8, 10 and 12 bitsBoth planes are computed from the same sample, so read it once and branch at the end:
const v = signal[this.thread.x];
const step = 2 / Math.pow(2, bits);
const q = Math.round(v / step) * step;
let out = q - v;
if (this.thread.y === 0) {
out = q;
}
return out;Sum the squares of the samples and the squares of plane 1, then
const snr = 10 * Math.log10(sigPower / errPower);
— and print it with the bit depth beside it so the ~6 dB step is readable.
f16 half-precision path is this exact
bargain — half the bandwidth and half the memory for a coarser step. Modern inference hardware
pushes it much further with int8 and int4 kernels, and the reason quantisation-aware training
works at all is that the error stays small, zero-mean noise instead of a bias.
This page is an interactive exercise — the editor, the GPU runner and your saved progress need JavaScript. The text above is the full brief.