Task 1 of 5

The Loss You Can Check

Fitting is optimisation. Pick a model — here the straight line y = m·x + c — pick a single number that says how badly it fits, then go looking for the parameters that make that number small. The number is the loss; for least squares it is the mean squared residual. Gradient descent has a reputation as the engine inside model training, but underneath it is nothing more than a numerical method for walking downhill, and that is all this module asks of it.

Look at what the loss actually is:

L(m, c) = (1/n) · Σ (m·xᵢ + c − yᵢ)²

A sum over every data point, divided by n — a reduction, the same many-in-one-out shape the Reductions module builds its halving ladder for. So the first kernel here is a partial-sum kernel with the squaring fused into the read: 64 threads, 64 points each, one pass over memory.

These 4,096 points were built so that every claim in this module is checkable to the last digit. The best-fitting line is exactly y = 3x + 4, the lowest reachable loss is exactly 0.5, and the whole surface is L(m, c) = (m − 3)² + (c − 4)² + 0.5. From m = c = 0, then, the loss must read 25.5.

Goal: finish the partial-sum kernel so it accumulates squared residuals, then divide the grand total by 4,096 and log the loss at m = 0, c = 0 — it should be 25.5.

Requirements

Each of the 64 threads walks its strided slice: i * this.constants.threads + this.thread.x
The residual of point at is m * xs[at] + c - ys[at]
Accumulate its square — squaring happens as the value is read, not in a second pass
Divide the total of the 64 partials by 4096 before logging

Hint 1 — read once, square immediately

Name the residual, then square the name — no second pass over the data:

const r = m * xs[at] + c - ys[at];
sum += r * r;

Hint 2 — mean, not total

The 64 partials add up to Σ r² over all 4,096 points. The loss is the mean squared residual, so the last step is total / 4096. Forget it and you get 104,448 instead of 25.5.

Same idea elsewhere

Every training loop on every platform begins with exactly this: a per-example loss, summed and averaged. CUB and Thrust reduce it with a tree, a WGSL compute shader does it with workgroup shared memory and subgroup adds. Fusing the square into the read rather than running a separate squaring pass is thrust::transform_reduce in one line, and it halves the memory traffic.

All tasks in Gradient Descent

This page is an interactive exercise — the editor, the GPU runner and your saved progress need JavaScript. The text above is the full brief.