# Averaging Across Time

*Task 2 of 6 · [Video Filters](https://gpu.rocks/learn/video-filters-4d39e404.md) · GPU.js Learn*

A single image can only be denoised by borrowing from its *neighbours in space*
— that is what a blur is, and it costs you detail. Video hands you a second axis for free.
The pixel at (12, 40) is being measured sixty times a second, and the scene is not changing
that fast; the noise is. Average a pixel with *itself* across frames and the noise
falls away while the edges stay exactly where they were.

The cheap way to do it is a **running average**, one line long and with no
history to store:

```js
avg = (1 - alpha) * avg + alpha * now
```

Each frame nudges the average a little toward itself. With `alpha = 0.25` a
change takes a few frames to fully arrive — which is the trade: small `alpha`
denoises harder and smears motion into a comet tail, large `alpha` keeps motion
crisp and keeps the noise with it.

Structurally this is the feedback loop from Pipelines & Textures: the kernel reads the
texture it is about to replace. `immutable: true` is what makes that legal —
every call renders to a *fresh* texture, so last frame's average is safe to read while
this frame's is being written. Leave it out and gpu.js stops you with the reason; that is the
library refusing to let you read a half-written buffer.

**Array layout in gpu.js**
Image data comes in row-major: `image[y][x]` is the pixel in row *y*,
column *x*, and each pixel is an `[r, g, b, a]` array with channels from
0 to 1. Mind the inversion that catches everyone — sizes are given width-first
(`output: [width, height]`), but indexing runs row-first, so this thread's own
pixel is `image[this.thread.y][this.thread.x]`. Swap those two and you read the
transpose of your image. Three-dimensional data follows the same rule:
`output: [w, h, d]` is indexed `[z][y][x]`.

## Figures

- **each frame folds into what the last one left behind — state outlives the frame**

## Goal

**Goal:** finish the `blend` kernel — `(1 − alpha)`
of the previous average plus `alpha` of this frame's luminance — and make the
feedback loop legal with `immutable: true`.

## Requirements

- Add `immutable: true` to `blend` (keep `pipeline: true`)
- Compute this frame's luminance inside `blend`: `0.299r + 0.587g + 0.114b`
- Return `(1 - alpha) * previous + alpha * now`, with `alpha` from `this.constants`

## Hint 1 — run it first

The starter throws, and the message names both the crime and the sentence: the
kernel's input and output are the same storage, and `immutable = true` is the
fix. gpu.js error messages are unusually honest.

## Hint 2 — which weight goes where

The new frame is the small contribution — it is one sample out of many. So
`alpha` multiplies `now`, and `1 - alpha` multiplies the
average you already had:

```js
return (1 - this.constants.alpha) * previous[this.thread.y][this.thread.x]
     + this.constants.alpha * now;
```

## Hint 3 — why the loop starts at 1

Frame 0 has no predecessor, so it cannot be blended with anything — it
*is* the starting average, which is what `seed` produces. The blending
starts at frame 1. Every stateful video filter has this line, and forgetting it is how
you get a garbage or NaN first frame.

## Same idea elsewhere

An exponential moving average over frames is the cheapest temporal filter there is,
and it is everywhere: TAA in game engines accumulates jittered samples into a history buffer
exactly like this, denoisers for real-time ray tracing blend the current noisy estimate into
an exponential history, and camera ISPs run one per pixel in hardware. The interesting part
of all of them is not this line — it is deciding when to *throw the history away*
because the scene moved.

## Starter code

```js
// Denoising along the time axis: average each pixel with itself.
const gpu = new GPU({ mode });

// The starting average is simply frame 0's luminance — nothing to blend with yet.
const seed = gpu.createKernel(function (frame) {
  const p = frame[this.thread.y][this.thread.x];
  return 0.299 * p[0] + 0.587 * p[1] + 0.114 * p[2];
}, { output: [64, 64], pipeline: true });

// One frame folded into the running average.
const blend = gpu.createKernel(function (frame, previous) {
  const p = frame[this.thread.y][this.thread.x];
  const now = 0.299 * p[0] + 0.587 * p[1] + 0.114 * p[2];
  // TODO: return (1 - alpha) of the previous average plus alpha of `now`.
  return previous[this.thread.y][this.thread.x];
}, {
  output: [64, 64],
  pipeline: true,
  constants: { alpha: 0.25 },
  // TODO: this kernel reads the very texture it is writing. Run it and let
  // the error message tell you the missing setting.
});

let state = await seed(frames[0]);
for (let i = 1; i < frames.length; i++) {
  state = await blend(frames[i], state); // last frame's output, straight back in
}

const smoothed = state.toArray ? await state.toArray() : state;
console.log('smoothed center:', smoothed[32][32].toFixed(4));
```

---

Interactive version: https://gpu.rocks/learn/video-filters-4d39e404/2

[Previous task](https://gpu.rocks/learn/video-filters-4d39e404/1.md) · [Next task](https://gpu.rocks/learn/video-filters-4d39e404/3.md)