Building Browser-Only Object Outlines with Detection, Prompted Segmentation, and Optical Flow

Introduction

I recently added an object-outline effect to Timeline Studio, an open-source browser-based video editor. The feature can isolate objects such as shoes, products, or toys and render a reusable outline around them—without uploading the source image or video to a server.

The complete implementation is available on GitHub:

https://github.com/MartinDelophy/ai-video-editor

This post focuses on three engineering decisions:

separating object detection from pixel-level segmentation;
keeping the same object identity across video frames;
rejecting unreliable masks instead of propagating bad results.

The processing pipeline

A conventional edge detector is not enough. It also picks up background texture, shadows, and internal details. I split the workflow into distinct stages:

Detect candidate objects
        ↓
Select and lock the editing target
        ↓
Create a prompt point inside the target
        ↓
Generate a pixel-level mask
        ↓
Validate mask quality
        ↓
Cache alpha and render the outline

The detector answers where the object is. Prompted segmentation answers which pixels belong to it. Keeping these responsibilities separate makes target selection more stable in crowded scenes.

Run inference outside the UI thread

Model inference on the main thread can freeze React controls and timeline interaction. Detection runs in a Web Worker, while the UI matches asynchronous responses by request ID.

worker.postMessage({
  requestId,
  type: "detect",
  blob,
  scoreThreshold: 0.24,
});

The ONNX Runtime session prefers WebGPU and falls back to WASM when GPU initialization is unavailable or fails:

async function createSession(modelBytes) {
  if (navigator.gpu) {
    try {
      return await ort.InferenceSession.create(modelBytes, {
        executionProviders: ["webgpu"],
        graphOptimizationLevel: "all",
      });
    } catch (error) {
      console.warn("WebGPU failed; falling back to WASM");
    }
  }

  return ort.InferenceSession.create(modelBytes, {
    executionProviders: ["wasm"],
    graphOptimizationLevel: "all",
  });
}

This keeps the fast path on supported devices without making WebGPU a hard requirement.

Selecting and locking the target

Choosing the highest-confidence detection on every frame can make the effect jump to another object. I rank candidates using confidence, area, distance from the center, overlap with the previous target, and class continuity.

function calculateRank(item, previous) {
  const area = getArea(item.box);
  const centered = getCenterScore(item.box);
  const previousIoU = boxIoU(item.box, previous?.box);
  const identityBonus = previous?.label === item.label ? 0.5 : 0;

  return (
    item.score * 0.55 +
    Math.min(area, 0.25) +
    centered * 0.2 +
    previousIoU * 1.2 +
    identityBonus
  );
}

For video, overlap with the previously accepted box receives a larger weight. A new object entering the frame should not silently become the editing target.

Prompted segmentation and alpha generation

Once the target is locked, I convert its center into a normalized prompt point and pass it to the segmentation model.

const result = segmenter.segment(image, {
  keypoint: {
    x: clamp(point.x, 0, 1),
    y: clamp(point.y, 0, 1),
  },
});

const probabilities = result.confidenceMasks.at(-1)
  .getAsFloat32Array();

const alpha = new Uint8ClampedArray(probabilities.length);
for (let i = 0; i < probabilities.length; i += 1) {
  const value = clamp(probabilities[i], 0, 1);
  alpha[i] = value >= threshold ? Math.round(value * 255) : 0;
}

The raw output is never trusted immediately. I validate mask area, connected components, target-box overflow, and changes relative to the previous accepted frame. A mask that includes background or another object is rejected.

Sparse anchors plus optical flow

Running detection and segmentation on every video frame is expensive in a browser. Full analysis runs only on sparse anchor frames; optical flow propagates the alpha between anchors.

const needsAnchor =
  index === 0 ||
  index - lastAnchorIndex >= anchorInterval ||
  !trackedMaskBox;

if (needsAnchor) {
  currentAlpha = await analyzeObjectAnchor(frame);
  lastAnchorIndex = index;
} else {
  currentAlpha = await propagateAlphaWithFlow(
    previousFrame,
    currentFrame,
    currentAlpha,
  );
}

If the propagated area changes implausibly, tracking is discarded and detection starts again:

const areaRatio = nextArea / previousArea;
if (!nextBox || areaRatio < 0.35 || areaRatio > 2.8) {
  currentAlpha = null;
  trackedMaskBox = null;
}

Failing closed matters here. Showing no effect for one uncertain frame is safer than legitimizing a wrong mask and carrying it forward.

Separate AI analysis from style rendering

Analysis is expensive, but changing color, width, glow, or material should feel immediate. I store the analyzed alpha separately from the effect style:

const objectOutline = {
  targetKind: "object",
  outline: {
    color: "#f3efe4",
    width: 12,
    opacity: 1,
    glow: 0.35,
  },
  material: {
    id: "paper",
    edgeDensity: 0.5,
    shadowDepth: 0.32,
  },
};

Style edits recompose the cached alpha instead of rerunning the models. Preview and export consume the same alpha and effect state, which reduces visual differences between the editor and the final video.