# Take the Step

*Task 3 of 5 · [Gradient Descent](https://gpu.rocks/learn/gradient-descent-c94c3f22.md) · GPU.js Learn*

The whole algorithm, now: stand somewhere, ask which way is up, move the other
way, repeat.

```js
m ← m − η · ∂L/∂m
c ← c − η · ∂L/∂c
```

η is the **learning rate** — how far you move per step. Both kernels
below are already written (task 1's loss, task 2's gradient), and the driving loop lives
in JavaScript, exactly as the halving ladder's driver does: the arithmetic that touches
all 4,096 points stays on the GPU, and the two numbers that say where you are stay on
the host.

Sixty steps at `η = 0.1` from `(0, 0)` land on
`(3, 4)` to five decimal places, and the loss falls
`25.5 → 0.79 → 0.503 → 0.5` along the way. Watch where it stops:
**0.5**, not zero. The scatter in this data is real, and no straight line
can explain it away — the optimum is where the loss stops falling, not where it
vanishes.

## Goal

**Goal:** write the two lines that move `m` and
`c` one step downhill, and land on `y = 3x + 4` with a final loss
of `0.5`.

## Requirements

- Step *against* the gradient — subtract, do not add
- Scale each step by `rate`: `m = m - rate * g[0]`
- `gradientAt()` is async — `await` it, and use the averaged gradient it returns, not a raw sum
- The logged final slope and intercept should read `3` and `4`

## Hint 1 — which way is downhill?

The gradient points in the direction the loss *increases* fastest.
At `(0, 0)` it is `(−6, −8)`, so downhill is
`(+6, +8)`, and `m − 0.1·(−6) = 0.6` is the first step. That
minus sign is the whole difference between fitting and exploding.

## Hint 2 — the two lines

```js
m = m - rate * g[0];
c = c - rate * g[1];
```

Both parameters move on the *same* gradient — the one measured before either of
them changed.

## Same idea elsewhere

Host-driven, device-computed is how a real optimiser runs: a Python training
loop issues CUDA kernels one step at a time, a WebGPU trainer records one dispatch per
step. Parameters are small and data is not, so the parameters live where the control
flow is — and the 60 round trips you just paid are exactly the cost real frameworks
fight by fusing whole steps into one launch.

## Starter code

```js
// Both kernels are already written. The algorithm is yours.
const gpu = new GPU({ mode });

const gradPartials = gpu.createKernel(function (xs, ys, m, c) {
  let sm = 0;
  let sc = 0;
  for (let i = 0; i < this.constants.chunk; i++) {
    const at = i * this.constants.threads + this.thread.x;
    const r = m * xs[at] + c - ys[at];
    sm += r * xs[at];
    sc += r;
  }
  if (this.thread.y === 0) {
    return sm;
  }
  return sc;
}, { output: [64, 2], constants: { threads: 64, chunk: 64 } });

const lossPartials = gpu.createKernel(function (xs, ys, m, c) {
  let sum = 0;
  for (let i = 0; i < this.constants.chunk; i++) {
    const at = i * this.constants.threads + this.thread.x;
    const r = m * xs[at] + c - ys[at];
    sum += r * r;
  }
  return sum;
}, { output: [64], constants: { threads: 64, chunk: 64 } });

async function gradientAt(m, c) {
  const rows = await gradPartials(xs, ys, m, c);
  let sumMx = 0;
  let sumC = 0;
  for (let i = 0; i < 64; i++) {
    sumMx += rows[0][i];
    sumC += rows[1][i];
  }
  return [(2 * sumMx) / 4096, (2 * sumC) / 4096];
}

async function lossAt(m, c) {
  const partials = await lossPartials(xs, ys, m, c);
  let total = 0;
  for (let i = 0; i < 64; i++) total += partials[i];
  return total / 4096;
}

const rate = 0.1;
let m = 0;
let c = 0;

for (let step = 1; step <= 60; step++) {
  const g = await gradientAt(m, c);
  // TODO: move m and c one step DOWNHILL.
  // g[0] is dL/dm and g[1] is dL/dc — both point UPHILL.

  if (step % 10 === 0) console.log('step', step, '· loss', await lossAt(m, c));
}

console.log('fitted slope:', m);
console.log('fitted intercept:', c);
console.log('final loss:', await lossAt(m, c));
```

---

Interactive version: https://gpu.rocks/learn/gradient-descent-c94c3f22/3

[Previous task](https://gpu.rocks/learn/gradient-descent-c94c3f22/2.md) · [Next task](https://gpu.rocks/learn/gradient-descent-c94c3f22/4.md)