Task 2 of 5

The Aperture Problem

One equation, two unknowns, is not "nearly enough information". It is a line of answers. Ix·u + Iy·v + It = 0 is the equation of a straight line in the (u, v) plane, and every point on it fits this pixel's evidence equally well. Looking harder at the pixel will not narrow it down; the information is not there.

What is there is the component of the motion along the gradient. Perpendicular to the gradient the intensity does not change, so sliding that way is invisible. Pick the shortest vector on the line and you get the normal flow:

u = −It · Ix / (Ix² + Iy²)
v = −It · Iy / (Ix² + Iy²)

derivs here comes from a pair you can check by hand. The scene is a sawtooth ramp climbing 12 levels per step along the (1, 1) diagonal — so every edge in it runs anti-diagonally — and frame 2 is that scene moved 4 pixels to the right and 0 down. You know the answer. The pixel does not, and cannot.

the window cannot tell these apart — only the across-the-edge part is real — An anti-diagonal edge seen through a small circular aperture. The true motion arrow points right; the only recoverable component points down-right, at right angles to the edge. A dashed line through both arrowheads marks the whole family of motions that fit the same evidence.

Goal: compute the normal flow for every pixel — plane 0 is u, plane 1 is v — and watch it fail to find a motion you know exactly.

Requirements

Keep output: [64, 64, 2]: plane 0 is u, plane 1 is v
Read this pixel's three derivatives from derivs[0][y][x], derivs[1][y][x], derivs[2][y][x]
Return -It * Ix / (Ix*Ix + Iy*Iy) for plane 0 and -It * Iy / (Ix*Ix + Iy*Iy) for plane 1
Sign convention: (u, v) is where the content went, in pixels per frame — positive u is rightward, positive v is downward. That is the sign that makes Ix·u + Iy·v + It come out zero

Hint 1 — one pixel, no neighbours

Nothing is gathered here: thread (x, y) reads exactly three numbers and returns one. The whole body fits in six lines.

Hint 2 — name the squared gradient once

const ix = derivs[0][y][x];
const iy = derivs[1][y][x];
const it = derivs[2][y][x];
const g = ix * ix + iy * iy;

then plane 0 returns (-it * ix) / g and plane 1 returns (-it * iy) / g. This pair has a gradient at every pixel, so no guard is needed yet — task 3 is where that stops being true.

Same idea elsewhere

Normal flow is not a toy: it is what a single-pixel constraint can honestly give you, and it is the per-thread starting point every dense-flow implementation refines — CUDA's NVOF optical-flow engine, OpenCV's calcOpticalFlowFarneback, the motion-vector passes inside DLSS and FSR. All of them add an assumption on top; none of them can conjure the missing component out of one pixel.

All tasks in Optical Flow

This page is an interactive exercise — the editor, the GPU runner and your saved progress need JavaScript. The text above is the full brief.