Task 2 of 5

Slice, Window, Transform

The fix falls out of saying the problem aloud: stop transforming the whole signal. Chop it into short frames, transform each one on its own, and stand the answers side by side. That is the short-time Fourier transform, and the picture it makes — time across, frequency up, brightness for how much — is a spectrogram.

Two numbers describe the chopping and they are not the same number. Window is how much signal one frame sees (256 samples here, 62.5 ms). Hop is how far the window slides between frames (64 samples). Setting hop equal to the window puts the frames edge to edge, so anything straddling a boundary is split in half; setting hop larger leaves gaps the transform never looks at. Hop smaller — overlap — costs work and buys a smooth time axis: here every sample is seen by four different frames. One frame more: cut a slice out with scissors and it ends on a step, which the transform faithfully reports as energy at every frequency. Windowing & Spectral Leakage makes a module of that; here, just take the fix and multiply each frame by a Hann bell first.

The GPU shape is the good part: one thread per (frame, bin). No cell needs anything another cell computed, so the entire picture is one launch of 8,192 threads, each gathering its own 256 samples. output: [FRAMES, BINS] puts time on x and frequency on y — which is how the picture is drawn, and means the result reads spec[bin][frame].

one window position, one column; slide by the hop and do it again
Goal: build the spectrogram of signal — get the frame count right, then fill in the kernel body so each cell holds the windowed magnitude.

Requirements

Hint 1 — which sample is mine?

Frame f, tap t of that frame, is signal[f * hop + t]. The tap index t also drives the window and the angle — both are measured from the start of the frame, never from the start of the signal.

Hint 2 — the loop body
for (let t = 0; t < this.constants.win; t++) {
  const w = 0.5 - 0.5 * Math.cos((2 * Math.PI * t) / (this.constants.win - 1));
  const s = signal[start + t] * w;
  const angle = (-2 * Math.PI * bin * t) / this.constants.win;
  re += s * Math.cos(angle);
  im += s * Math.sin(angle);
}
return Math.sqrt(re * re + im * im) / this.constants.win;
Hint 3 — counting frames

signal.length / HOP counts frames that start inside the signal, including three at the end whose windows run off it. A frame needs a whole window, so the last legal start is signal.length - WIN:

const FRAMES = Math.floor((signal.length - WIN) / HOP) + 1;

With 4,288 samples that is 64, and the last frame ends exactly on the last sample.

Same idea elsewhere

You have written librosa.stft / torchaudio.Spectrogram / MATLAB's spectrogram, and the production version differs in exactly one place: the inner loop becomes an FFT and the whole thing becomes a batched transform — one cuFFT or VkFFT plan, all 64 frames at once. The 2D launch you just wrote is already the right decomposition; only the O(n) inner sum gets replaced by an O(log n) ladder. Note what you did not need: no atomics, no shared memory, no thread writing to a cell it does not own. Every thread pulled what it wanted. That is the gather formulation, and it is why this maps onto every platform unchanged.

All tasks in Spectrograms

  1. One Spectrum, No Clock
  2. Slice, Window, Transform
  3. Fine in Time, or Fine in Frequency
  4. Decibels, Then Colour
  5. Payoff: Read the Sweep Off the Picture

This page is an interactive exercise — the editor, the GPU runner and your saved progress need JavaScript. The text above is the full brief.