# Slice, Window, Transform

*Task 2 of 5 · [Spectrograms](https://gpu.rocks/learn/spectrograms-9ecd2295.md) · GPU.js Learn*

The fix falls out of saying the problem aloud: stop transforming the whole signal.
Chop it into short **frames**, transform each one on its own, and stand the
answers side by side. That is the **short-time Fourier transform**, and the
picture it makes — time across, frequency up, brightness for how much — is a spectrogram.

Two numbers describe the chopping and they are *not* the same number.
**Window** is how much signal one frame sees (256 samples here, 62.5 ms).
**Hop** is how far the window slides between frames (64 samples). Setting hop
equal to the window puts the frames edge to edge, so anything straddling a boundary is
split in half; setting hop larger leaves gaps the transform never looks at. Hop smaller —
**overlap** — costs work and buys a smooth time axis: here every sample is
seen by four different frames. One frame more: cut a slice out with scissors and it ends on
a step, which the transform faithfully reports as energy at every frequency. Windowing &
Spectral Leakage makes a module of that; here, just take the fix and multiply each frame by
a Hann bell first.

The GPU shape is the good part: **one thread per (frame, bin)**. No cell
needs anything another cell computed, so the entire picture is one launch of 8,192 threads,
each gathering its own 256 samples. `output: [FRAMES, BINS]` puts time on x and
frequency on y — which is how the picture is drawn, and means the result reads
`spec[bin][frame]`.

## Figures

- **one window position, one column; slide by the hop and do it again**

## Goal

**Goal:** build the spectrogram of `signal` — get the frame
count right, then fill in the kernel body so each cell holds the windowed magnitude.

## Requirements

- Set `FRAMES` to the number of *whole* windows that fit: `Math.floor((signal.length - WIN) / HOP) + 1`
- Frame `this.thread.x` starts at `this.thread.x * this.constants.hop` and reads `this.constants.win` samples
- Multiply each sample by its Hann value `0.5 - 0.5 * Math.cos(2 * Math.PI * t / (this.constants.win - 1))` before accumulating
- Return `Math.sqrt(re * re + im * im) / this.constants.win`

## Hint 1 — which sample is mine?

Frame `f`, tap `t` of that frame, is
`signal[f * hop + t]`. The tap index `t` also drives the window
and the angle — both are measured from the *start of the frame*, never from the
start of the signal.

## Hint 2 — the loop body

```js
for (let t = 0; t < this.constants.win; t++) {
  const w = 0.5 - 0.5 * Math.cos((2 * Math.PI * t) / (this.constants.win - 1));
  const s = signal[start + t] * w;
  const angle = (-2 * Math.PI * bin * t) / this.constants.win;
  re += s * Math.cos(angle);
  im += s * Math.sin(angle);
}
return Math.sqrt(re * re + im * im) / this.constants.win;
```

## Hint 3 — counting frames

`signal.length / HOP` counts frames that start inside the signal,
including three at the end whose windows run off it. A frame needs a *whole*
window, so the last legal start is `signal.length - WIN`:

```js
const FRAMES = Math.floor((signal.length - WIN) / HOP) + 1;
```

With 4,288 samples that is 64, and the last frame ends exactly on the last sample.

## Same idea elsewhere

You have written `librosa.stft` / `torchaudio.Spectrogram` /
MATLAB's `spectrogram`, and the production version differs in exactly one place:
the inner loop becomes an FFT and the whole thing becomes a *batched* transform —
one cuFFT or VkFFT plan, all 64 frames at once. The 2D launch you just wrote is already the
right decomposition; only the O(n) inner sum gets replaced by an O(log n) ladder. Note what
you did *not* need: no atomics, no shared memory, no thread writing to a cell it does
not own. Every thread pulled what it wanted. That is the gather formulation, and it is why
this maps onto every platform unchanged.

## Starter code

```js
// The short-time Fourier transform: one thread per (frame, bin).
const gpu = new GPU({ mode });

const WIN = 256;   // samples each frame sees — 62.5 ms at 4096 Hz
const HOP = 64;    // how far the window slides between frames
const BINS = 128;  // bins kept per frame: 0 … just below Nyquist

// TODO: how many WHOLE windows fit? A frame whose window runs off the end
// of the signal is not a frame.
const FRAMES = Math.floor(signal.length / HOP);

const spectrogram = gpu.createKernel(function (signal) {
  const frame = this.thread.x;
  const bin = this.thread.y;
  const start = frame * this.constants.hop;
  let re = 0;
  let im = 0;

  // TODO: walk this frame's this.constants.win samples.
  //   w     = 0.5 - 0.5 * Math.cos(2 * Math.PI * t / (this.constants.win - 1))
  //   s     = signal[start + t] * w
  //   angle = -2 * Math.PI * bin * t / this.constants.win
  //   re += s * Math.cos(angle);   im += s * Math.sin(angle);
  // Then return the magnitude, divided by this.constants.win.

  return 0;
}, {
  output: [FRAMES, BINS],
  constants: { win: WIN, hop: HOP },
});

const spec = spectrogram(signal);
console.log('spectrogram:', spec.length, 'bins ×', spec[0].length, 'frames');

// signal is a chirp — a tone sliding steadily upward — so the loudest bin
// should climb, one bin per frame.
for (const f of [0, 16, 32, 48]) {
  let best = 0;
  for (let b = 1; b < BINS; b++) if (spec[b][f] > spec[best][f]) best = b;
  console.log('frame', f, 'peaks at bin', best);
}
```

---

Interactive version: https://gpu.rocks/learn/spectrograms-9ecd2295/2

[Previous task](https://gpu.rocks/learn/spectrograms-9ecd2295/1.md) · [Next task](https://gpu.rocks/learn/spectrograms-9ecd2295/3.md)