# What Can We Afford to Lose?

*Task 1 of 6 · [Seam Carving: Content-Aware Resizing](https://gpu.rocks/learn/seam-carving-a23a0d9b.md) · GPU.js Learn*

To make a picture narrower you can squash it, crop it — or delete the pixels
nobody would miss. Seam carving does the third: it removes a **seam**, a
connected path of one pixel per row, threaded through the least interesting part of the
picture, and does it once per column you want to lose. Everything then slides across to
close the gap, so the interesting parts keep their proportions and the boring parts get
squeezed out.

"Interesting" needs a number, and the usual one is the **gradient
magnitude**: flat regions score near zero, edges score high. That is exactly the
Sobel pass *Convolution & Filters* already derives — the two weight grids are in
the starter and we are not deriving them again. What changes here is two small things, and
both matter later:

The energy must be defined **at the border**. A seam is allowed to run
straight down the edge of the picture, so column 0 needs a price like every other column —
clamp the neighbour coordinates instead of painting the border black. And the kernel must
work at **any width**, because after the first seam comes out the picture is
127 columns wide, then 126… so the size comes from `this.output.x` and
`this.output.y`, never from a constant.

**Array layout in gpu.js**
Image data comes in row-major: `image[y][x]` is the pixel in row *y*,
column *x*, and each pixel is an `[r, g, b, a]` array with channels from
0 to 1. Mind the inversion that catches everyone — sizes are given width-first
(`output: [width, height]`), but indexing runs row-first, so this thread's own
pixel is `image[this.thread.y][this.thread.x]`. Swap those two and you read the
transpose of your image. Three-dimensional data follows the same rule:
`output: [w, h, d]` is indexed `[z][y][x]`.

## Goal

**Goal:** finish `energy` so that cell `[y][x]`
holds `Math.sqrt(gx * gx + gy * gy)` for the 3×3 neighbourhood of
`gray`, with every neighbour coordinate clamped to the picture.

## Requirements

- Clamp all four neighbour coordinates — `Math.max(x - 1, 0)` and `Math.min(x + 1, this.output.x - 1)`, the same on `y` — so the border has an energy rather than a hole
- Compute `gx` (right column minus left) and `gy` (bottom row minus top) with the two Sobel grids in the starter
- Return the magnitude `Math.sqrt(gx * gx + gy * gy)`
- Keep `dynamicOutput: true` and `dynamicArguments: true` — the same kernel runs at every width the picture passes through

## Hint 1 — the clamp

Four one-liners, and they are the only thing standing between you and a NaN in
the first and last row and column:

```js
const xm = Math.max(x - 1, 0);
const xp = Math.min(x + 1, this.output.x - 1);
```

— and the same pair on `y`, against `this.output.y - 1`. A pixel on
the edge now simply sees itself twice, which is what "replicate the border" means.

## Hint 2 — the two sums

Read them straight off the grids in the starter — right column minus left,
middle row counted double:

```js
const gx = (gray[ym][xp] + 2 * gray[y][xp] + gray[yp][xp])
         - (gray[ym][xm] + 2 * gray[y][xm] + gray[yp][xm]);
```

`gy` is the same move on rows: bottom row minus top row, middle column
counted double.

## Hint 3 — why not a constant

`this.output.x` is the width of *this* launch, not of the
original picture. Hard-code `127` and the kernel is right exactly once —
the first time — and then quietly clamps to a column that no longer exists.

## Same idea elsewhere

Every content-aware tool starts by building a cost field and only then decides
what to do with it. NVIDIA's NPP and OpenCV's CUDA module both ship Sobel as a primitive;
a video encoder builds the same gradient field to decide which macroblocks deserve bits;
and "saliency map first, decision second" is the shape of seam carving, content-aware
fill and adaptive sampling alike.

## Starter code

```js
// Energy: how much does this pixel's neighbourhood change? Flat = cheap.
const gpu = new GPU({ mode });

// The picture arrives as ImageData. One pass turns it into luminance
// (module "Data In, Data Out" writes this one) — already done for you.
const luminance = gpu.createKernel(function (image) {
  const p = image[this.thread.y][this.thread.x];
  return 0.299 * p[0] + 0.587 * p[1] + 0.114 * p[2];
}, { output: [128, 72] });

// Sobel's two weight grids, exactly as Convolution & Filters derives them:
//
//        Gx              Gy
//    -1   0  +1      -1  -2  -1
//    -2   0  +2       0   0   0
//    -1   0  +1      +1  +2  +1
//
const energy = gpu.createKernel(function (gray) {
  const x = this.thread.x;
  const y = this.thread.y;
  // TODO 1: clamp the four neighbour coordinates to the picture. The width
  //         and height of THIS launch are this.output.x and this.output.y.
  const xm = x;
  const xp = x;
  const ym = y;
  const yp = y;
  // TODO 2: gx = right column - left column, gy = bottom row - top row,
  //         then return Math.sqrt(gx * gx + gy * gy).
  return gray[y][x];
}, {
  output: [128, 72],
  dynamicOutput: true,   // the picture narrows every time a seam comes out
  dynamicArguments: true,
});

// A look at what we built: bright where the picture is busy.
const paint = gpu.createKernel(function (e) {
  // thread.y 0 is the BOTTOM row of a canvas, so read the rows in reverse
  const v = e[this.output.y - 1 - this.thread.y][this.thread.x];
  this.color(v, v, v, 1);
}, { output: [128, 72], graphical: true });

const gray = await luminance(photo);
const map = await energy(gray);
await paint(map);
render(paint.canvas);

// A logged numeric array draws its own sparkline: this is one row of energy,
// left to right — flat corridor, texture, two poles, and the face.
console.log('energy across row 20:', map[20]);
```

---

Interactive version: https://gpu.rocks/learn/seam-carving-a23a0d9b/1

[Next task](https://gpu.rocks/learn/seam-carving-a23a0d9b/2.md)