# The Loss You Can Check

*Task 1 of 5 · [Gradient Descent](https://gpu.rocks/learn/gradient-descent-c94c3f22.md) · GPU.js Learn*

Fitting is optimisation. Pick a model — here the straight line
`y = m·x + c` — pick a single number that says how badly it fits, then go
looking for the parameters that make that number small. The number is the
**loss**; for least squares it is the mean squared residual. Gradient
descent has a reputation as the engine inside model training, but underneath it is
nothing more than a numerical method for walking downhill, and that is all this
module asks of it.

Look at what the loss actually *is*:

```js
L(m, c) = (1/n) · Σ (m·xᵢ + c − yᵢ)²
```

A sum over every data point, divided by n — a **reduction**, the same
many-in-one-out shape the Reductions module builds its halving ladder for. So the first
kernel here is a partial-sum kernel with the squaring fused into the read: 64 threads,
64 points each, one pass over memory.

These 4,096 points were built so that every claim in this module is checkable to the
last digit. The best-fitting line is exactly `y = 3x + 4`, the lowest
reachable loss is exactly `0.5`, and the whole surface is
`L(m, c) = (m − 3)² + (c − 4)² + 0.5`. From `m = c = 0`, then,
the loss must read `25.5`.

## Goal

**Goal:** finish the partial-sum kernel so it accumulates
**squared** residuals, then divide the grand total by 4,096 and log the
loss at `m = 0, c = 0` — it should be `25.5`.

## Requirements

- Each of the 64 threads walks its strided slice: `i * this.constants.threads + this.thread.x`
- The residual of point `at` is `m * xs[at] + c - ys[at]`
- Accumulate its *square* — squaring happens as the value is read, not in a second pass
- Divide the total of the 64 partials by 4096 before logging

## Hint 1 — read once, square immediately

Name the residual, then square the name — no second pass over the data:

```js
const r = m * xs[at] + c - ys[at];
sum += r * r;
```

## Hint 2 — mean, not total

The 64 partials add up to `Σ r²` over all 4,096 points. The loss
is the *mean* squared residual, so the last step is
`total / 4096`. Forget it and you get 104,448 instead of 25.5.

## Same idea elsewhere

Every training loop on every platform begins with exactly this: a per-example
loss, summed and averaged. CUB and Thrust reduce it with a tree, a WGSL compute shader
does it with workgroup shared memory and subgroup adds. Fusing the square into the read
rather than running a separate squaring pass is `thrust::transform_reduce`
in one line, and it halves the memory traffic.

## Starter code

```js
// 4,096 points, 64 threads, 64 points each — strided, so neighbouring
// threads read neighbouring points at every step of the loop.
const gpu = new GPU({ mode });

const lossPartials = gpu.createKernel(function (xs, ys, m, c) {
  let sum = 0;
  for (let i = 0; i < this.constants.chunk; i++) {
    const at = i * this.constants.threads + this.thread.x;
    // TODO: the residual of point `at` is m * xs[at] + c - ys[at].
    // Accumulate its SQUARE into sum.
    sum += 0;
  }
  return sum;
}, {
  output: [64],
  constants: { threads: 64, chunk: 64 },
});

const partials = await lossPartials(xs, ys, 0, 0);

let total = 0;
for (let i = 0; i < partials.length; i++) total += partials[i];

// TODO: the loss is the MEAN squared residual — divide by 4096.
console.log('loss at m = 0, c = 0:', total);
```

---

Interactive version: https://gpu.rocks/learn/gradient-descent-c94c3f22/1

[Next task](https://gpu.rocks/learn/gradient-descent-c94c3f22/2.md)