Task 3 of 5

Take the Step

The whole algorithm, now: stand somewhere, ask which way is up, move the other way, repeat.

m ← m − η · ∂L/∂m
c ← c − η · ∂L/∂c

η is the learning rate — how far you move per step. Both kernels below are already written (task 1's loss, task 2's gradient), and the driving loop lives in JavaScript, exactly as the halving ladder's driver does: the arithmetic that touches all 4,096 points stays on the GPU, and the two numbers that say where you are stay on the host.

Sixty steps at η = 0.1 from (0, 0) land on (3, 4) to five decimal places, and the loss falls 25.5 → 0.79 → 0.503 → 0.5 along the way. Watch where it stops: 0.5, not zero. The scatter in this data is real, and no straight line can explain it away — the optimum is where the loss stops falling, not where it vanishes.

Goal: write the two lines that move m and c one step downhill, and land on y = 3x + 4 with a final loss of 0.5.

Requirements

Hint 1 — which way is downhill?

The gradient points in the direction the loss increases fastest. At (0, 0) it is (−6, −8), so downhill is (+6, +8), and m − 0.1·(−6) = 0.6 is the first step. That minus sign is the whole difference between fitting and exploding.

Hint 2 — the two lines
m = m - rate * g[0];
c = c - rate * g[1];

Both parameters move on the same gradient — the one measured before either of them changed.

Same idea elsewhere

Host-driven, device-computed is how a real optimiser runs: a Python training loop issues CUDA kernels one step at a time, a WebGPU trainer records one dispatch per step. Parameters are small and data is not, so the parameters live where the control flow is — and the 60 round trips you just paid are exactly the cost real frameworks fight by fusing whole steps into one launch.

All tasks in Gradient Descent

  1. The Loss You Can Check
  2. The Gradient Is a Reduction
  3. Take the Step
  4. How Big a Step? Ask 256 at Once
  5. A Thousand Starts, Three Answers

This page is an interactive exercise — the editor, the GPU runner and your saved progress need JavaScript. The text above is the full brief.