# The Gradient Is a Reduction

*Task 2 of 5 · [Gradient Descent](https://gpu.rocks/learn/gradient-descent-c94c3f22.md) · GPU.js Learn*

To walk downhill you need the direction of steepest *ascent* — the
**gradient** — and then you go the other way. Differentiating the loss is
two lines of calculus, and for a model this small there is no reason to reach for
anything cleverer than writing them out:

```js
rᵢ      = m·xᵢ + c − yᵢ
∂L/∂m  = (2/n) · Σ rᵢ · xᵢ
∂L/∂c  = (2/n) · Σ rᵢ
```

Two sums, over the same residuals, in the same order. There is no reason to walk the
data twice: one kernel computes both, holding each residual in a register and pushing it
into two accumulators. What comes back is a 2D output — `output: [64, 2]`,
64 threads wide and 2 rows tall — where **row 0** holds the
`Σ r·x` partials and **row 1** the `Σ r` partials.

Notice where the branch that picks the row goes: *after* the loop, never
inside it. Every thread then runs the identical straight-line body and only the last
statement differs. At `m = c = 0` the gradient must come out
`(−6, −8)`, because on this data `∇L = (2(m − 3), 2(c − 4))`.

## Figures

- **read each residual once, spend it twice**

## Goal

**Goal:** fill in the two accumulators and the row selection, then
assemble the gradient in JavaScript and log it — `(−6, −8)` at
`m = 0, c = 0`.

## Requirements

- `sm` accumulates `r * xs[at]`, `sc` accumulates `r`
- Row 0 of the output returns `sm`, row 1 returns `sc` — branch on `this.thread.y` *after* the loop
- Total each row in JavaScript and multiply by `2 / 4096`
- Log both components

## Hint 1 — one residual, two homes

Read the residual once and use it twice, exactly as the fused loss kernel
reused it:

```js
const r = m * xs[at] + c - ys[at];
sm += r * xs[at];
sc += r;
```

## Hint 2 — which row am I?

`this.thread.y` is 0 for the first row and 1 for the second. Put
the choice after the loop so the loop body stays identical for every thread:

```js
if (this.thread.y === 0) {
  return sm;
}
return sc;
```

## Hint 3 — the 2/n out front

A 2D result is indexed `rows[y][x]`, so `rows[0]` is
the 64 slope partials and `rows[1]` the 64 intercept partials. Both
totals then need the same scaling: `(2 * total) / 4096`.

## Same idea elsewhere

Accumulating several statistics in one pass over the data is standard practice
everywhere: a CUDA kernel keeps both partials in registers across a single grid-stride
loop, CUB instantiates one `BlockReduce` per accumulator but reads the tile
once, and WGSL does the same with two workgroup arrays. The 2D output is just gpu.js's
spelling of *one launch, two results*.

## Starter code

```js
// One walk over the data, two sums out of it.
// Row 0 → the Σ r·x partials, row 1 → the Σ r partials.
const gpu = new GPU({ mode });

const gradPartials = gpu.createKernel(function (xs, ys, m, c) {
  let sm = 0;
  let sc = 0;
  for (let i = 0; i < this.constants.chunk; i++) {
    const at = i * this.constants.threads + this.thread.x;
    const r = m * xs[at] + c - ys[at];
    // TODO: sm accumulates the residual weighted by x, sc the residual itself.
    sm += 0;
    sc += 0;
  }
  // TODO: row 0 of the output is sm, row 1 is sc. Branch on this.thread.y.
  return sm;
}, {
  output: [64, 2],
  constants: { threads: 64, chunk: 64 },
});

const rows = await gradPartials(xs, ys, 0, 0);

let sumMx = 0;
let sumC = 0;
for (let i = 0; i < 64; i++) {
  sumMx += rows[0][i];
  sumC += rows[1][i];
}

// TODO: both components carry a factor of 2/n. n is 4096.
const gm = sumMx;
const gc = sumC;
console.log('gradient at m = 0, c = 0:', gm, gc);
```

---

Interactive version: https://gpu.rocks/learn/gradient-descent-c94c3f22/2

[Previous task](https://gpu.rocks/learn/gradient-descent-c94c3f22/1.md) · [Next task](https://gpu.rocks/learn/gradient-descent-c94c3f22/3.md)