Task 2 of 6

Averaging Across Time

A single image can only be denoised by borrowing from its neighbours in space — that is what a blur is, and it costs you detail. Video hands you a second axis for free. The pixel at (12, 40) is being measured sixty times a second, and the scene is not changing that fast; the noise is. Average a pixel with itself across frames and the noise falls away while the edges stay exactly where they were.

The cheap way to do it is a running average, one line long and with no history to store:

avg = (1 - alpha) * avg + alpha * now

Each frame nudges the average a little toward itself. With alpha = 0.25 a change takes a few frames to fully arrive — which is the trade: small alpha denoises harder and smears motion into a comet tail, large alpha keeps motion crisp and keeps the noise with it.

Structurally this is the feedback loop from Pipelines & Textures: the kernel reads the texture it is about to replace. immutable: true is what makes that legal — every call renders to a fresh texture, so last frame's average is safe to read while this frame's is being written. Leave it out and gpu.js stops you with the reason; that is the library refusing to let you read a half-written buffer.

Array layout in gpu.js

Image data comes in row-major: image[y][x] is the pixel in row y, column x, and each pixel is an [r, g, b, a] array with channels from 0 to 1. Mind the inversion that catches everyone — sizes are given width-first (output: [width, height]), but indexing runs row-first, so this thread's own pixel is image[this.thread.y][this.thread.x]. Swap those two and you read the transpose of your image. Three-dimensional data follows the same rule: output: [w, h, d] is indexed [z][y][x].

each frame folds into what the last one left behind — state outlives the frame
Goal: finish the blend kernel — (1 − alpha) of the previous average plus alpha of this frame's luminance — and make the feedback loop legal with immutable: true.

Requirements

Hint 1 — run it first

The starter throws, and the message names both the crime and the sentence: the kernel's input and output are the same storage, and immutable = true is the fix. gpu.js error messages are unusually honest.

Hint 2 — which weight goes where

The new frame is the small contribution — it is one sample out of many. So alpha multiplies now, and 1 - alpha multiplies the average you already had:

return (1 - this.constants.alpha) * previous[this.thread.y][this.thread.x]
     + this.constants.alpha * now;
Hint 3 — why the loop starts at 1

Frame 0 has no predecessor, so it cannot be blended with anything — it is the starting average, which is what seed produces. The blending starts at frame 1. Every stateful video filter has this line, and forgetting it is how you get a garbage or NaN first frame.

Same idea elsewhere

An exponential moving average over frames is the cheapest temporal filter there is, and it is everywhere: TAA in game engines accumulates jittered samples into a history buffer exactly like this, denoisers for real-time ray tracing blend the current noisy estimate into an exponential history, and camera ISPs run one per pixel in hardware. The interesting part of all of them is not this line — it is deciding when to throw the history away because the scene moved.

All tasks in Video Filters

  1. Sixteen Milliseconds
  2. Averaging Across Time
  3. What Moved?
  4. Learning the Empty Room
  5. The Payoff: A Virtual Background
  6. Going Live, Honestly

This page is an interactive exercise — the editor, the GPU runner and your saved progress need JavaScript. The text above is the full brief.