Task 1 of 6

Sixteen Milliseconds

Everything so far in this track processed one picture. Video changes the economics completely. At 60 frames per second you get 16.7 milliseconds to do all of it — decode, filter, composite, present — and then the frame is gone whether you were finished or not. Miss the budget and you do not get a slower filter, you get a stuttering one.

The same three kernels now run eight times instead of once, which turns two habits that were merely wasteful into the whole problem. The first is creating kernels inside the loop: createKernel transpiles your JavaScript to shader source and compiles it, so doing it per frame pays the compiler sixty times a second. The second is the readback — Pipelines & Textures showed the mechanism, and here it is the difference between a filter that runs and one that does not.

The starter below is honest, working, and unshippable. Fix its structure, then read the numbers it prints — and hit ⏱ Benchmark afterwards to see the same argument made twice.

Array layout in gpu.js

Image data comes in row-major: image[y][x] is the pixel in row y, column x, and each pixel is an [r, g, b, a] array with channels from 0 to 1. Mind the inversion that catches everyone — sizes are given width-first (output: [width, height]), but indexing runs row-first, so this thread's own pixel is image[this.thread.y][this.thread.x]. Swap those two and you read the transpose of your image. Three-dimensional data follows the same rule: output: [w, h, d] is indexed [z][y][x].

same three kernels, same arithmetic — the readbacks are what miss the frame

Goal: hoist the three kernels out of the frame loop, keep the two intermediate stages on the GPU, and report the per-frame cost against the 16.7 ms budget.

Requirements

Create the three kernels once, above the loop — exactly three for the whole run
Give luminance and denoise pipeline: true; tone stays plain
Process every frame in frames, with no .toArray() inside the loop
Log 'processed', out.length, 'frames' and a per-frame line carrying the ms and the fps

Hint 1 — what belongs in the loop

Everything that depends on which frame you are looking at, and nothing else. A kernel does not depend on the frame — it takes one as an argument. Three createKernel calls, then a loop that does nothing but call them.

Hint 2 — where the readbacks are hiding

There is no .toArray() in the starter, and the readbacks are still there: a non-pipeline kernel's return value is the readback. Add pipeline: true to the first two stages and the chain collapses to one expression:

out.push(await tone(await denoise(await luminance(frames[i]))));

Hint 3 — the budget arithmetic

One frame at 60 fps is 1000 / 60 = 16.7 ms. So:

const perFrame = totalMs / frames.length;
const fps = 1000 / perFrame;

and the verdict is just perFrame <= 16.7.

Same idea elsewhere

Every real-time GPU API separates "build the pipeline" from "run it", precisely so the expensive half happens once: WebGPU's createRenderPipeline versus dispatchWorkgroups, Vulkan's pipeline objects, CUDA modules loaded once and launched forever. And the per-frame budget is why frame graphs exist at all — a readback mid-frame is a full pipeline stall on every one of them.

All tasks in Video Filters

This page is an interactive exercise — the editor, the GPU runner and your saved progress need JavaScript. The text above is the full brief.