Three Phases of AI B-Roll: From Manual Beats to Vision-Authored Cutaways

B-roll is what makes a UGC testimonial feel edited rather than generated. The presenter talks to camera; cutaways show the product in hand, the label close-up, the lifestyle context. Without b-roll, AI ads read as a single static talking head — technically correct, emotionally flat.

On the AI ad platform, b-roll capability did not arrive fully formed. It evolved through three deliberate phases inside the tool's scene planner, each solving the limitations of the last. Phase 1 gave editors control. Phase 2 automated timing from references. Phase 3 generated the cutaways themselves with vision-authored prompts. A power-user feature flag gated the progression so we could ship incrementally without breaking production swipe flows.

B-roll is not decoration — it is narrative punctuation. The pipeline had to learn where to punctuate before it could learn what to show.

Why b-roll needed its own evolution arc

Main-scene generation and b-roll generation are different problems. Person scenes need identity anchoring, motion transfer, and VO sync. B-roll cutaways need product visibility, pacing that matches the A-roll beat, and — critically — no competing human faces that confuse the viewer about who the protagonist is.

Early swipe iterations treated b-roll as an afterthought: if the reference had cutaways, copy the timestamps; if not, skip them. Script-first UGC mode made that insufficient. Editors writing original scripts still needed cutaways, but had no reference timing to swipe. The three-phase arc was the structured response.

Phase 1: Manual b-roll beat placement

Phase 1 shipped inside the tool's scene planner as cutaway beat markers. Editors scrubbed the A-roll timeline and dropped b-roll insertion points — the same mental model as marking ad breaks in a non-linear editor, simplified to click-to-insert.

interface BrollBeat {
  id: string;
  insertAfterSceneId: string;
  offsetMs: number;           // within-scene offset
  durationMs: number;         // target cutaway length
  brief?: string;             // optional editor hint
  source: 'manual';
}

function insertManualBeat(
  plan: ScenePlan,
  afterSceneId: string,
  offsetMs: number,
): ScenePlan {
  const beat: BrollBeat = {
    id: generateId(),
    insertAfterSceneId: afterSceneId,
    offsetMs,
    durationMs: DEFAULT_BROLL_DURATION_MS,
    source: 'manual',
  };
  return { ...plan, brollBeats: [...plan.brollBeats, beat] };
}

Phase 1 proved the data model. Every subsequent phase reuses BrollBeat — only the source field and prompt generation logic change. Beats could be inserted between scenes (not just mid-scene), which mattered for script-first runs where scene boundaries align with script paragraphs rather than reference cuts.

The limitation was obvious: manual placement scales poorly. A twelve-scene ad with three cutaways each meant thirty-six click decisions per run. Editors wanted the system to propose timing, not just slots.

Phase 2: Auto-placement from reference timing

Phase 2 activated when a reference video existed. The analyzer extracted b-roll segments from the reference — moments where the camera cut away from the presenter to product or lifestyle footage — and the planner mapped those timestamps onto the generated A-roll timeline.

The "swipe" metaphor is literal: b-roll timing swipes from reference to generated plan, adjusted for duration differences between reference VO and generated VO.

interface ReferenceBrollSegment {
  startMs: number;
  endMs: number;
  classification: 'product' | 'lifestyle' | 'detail';
}

function swipeBrollTiming(
  referenceSegments: ReferenceBrollSegment[],
  generatedPlan: ScenePlan,
  referenceDurationMs: number,
  generatedDurationMs: number,
): BrollBeat[] {
  const scale = generatedDurationMs / referenceDurationMs;
  return referenceSegments.map(seg => ({
    id: generateId(),
    insertAfterSceneId: mapTimestampToScene(seg.startMs * scale, generatedPlan),
    offsetMs: sceneLocalOffset(seg.startMs * scale, generatedPlan),
    durationMs: (seg.endMs - seg.startMs) * scale,
    classification: seg.classification,
    source: 'reference-swipe',
  }));
}

Phase 2 dramatically reduced editor labor on reference-first runs. It did nothing for script-first runs — which is exactly why Phase 3 existed.

Phase 3: AI-generated cutaways with vision-authored prompts

Phase 3 is the full solution: the system generates b-roll clips, not just timing slots. The pipeline:

1. Beat proposal — LLM reads the script and proposes cutaway moments (product reveal, usage demo, lifestyle context)
2. Vision-authored prompts — vision model inspects the adjacent A-roll frame and writes a b-roll prompt grounded in what is actually visible
3. Generation — cutaway renders through the video model with people-free constraints
4. Editor brief — optional override text merged into the vision prompt
5. Cloned-voice VO — b-roll segments can carry supplementary narration when the script calls for it

async function authorBrollPrompt(
  aRollFrame: Buffer,
  beat: BrollBeat,
  product: ProductCatalogEntry,
  editorBrief?: string,
): Promise<string> {
  const visionContext = await visionModel.describe({
    frame: aRollFrame,
    focus: ['visible product', 'setting', 'lighting', 'palette'],
    exclude: ['faces', 'presenter identity'],
  });

  return promptComposer.compose({
    template: 'broll-cutaway',
    visionContext,
    productLock: product.heroImageUrl,
    editorBrief,
    constraints: PEOPLE_FREE_CONSTRAINTS,
  });
}

Vision-authored prompts solved a subtle quality problem. Text-only b-roll prompts hallucinate context — "woman in sunny kitchen holding product" when the A-roll frame shows a bathroom vanity with cool lighting. Grounding the prompt in the actual frame produces cutaways that match the established scene geography.

People-free b-roll: identity protection

The most important constraint in Phase 3 is people-free b-roll. If a cutaway shows a human face — even a generic stock-looking person — viewers subconsciously reassign protagonist identity. Scene four's presenter no longer matches scene seven's because the b-roll face became a competing anchor.

People-free constraints apply at three layers:

• Prompt negatives — explicit exclusion of faces, hands with identifiable features, and full human figures
• Vision QA — post-generation frame scan rejects clips with detected faces above confidence threshold
• Product-only fallback — if people-free generation fails twice, fall back to product hero animation without lifestyle context

The captioned Google Drive link fix addressed a specific bug: editors pasted Drive URLs to reference b-roll footage, but the link preview showed the wrong thumbnail. We added caption validation and people-free checks on uploaded reference clips before they entered the prompt chain.

Feature flag gating and progressive rollout

All three phases coexist behind a power flag — UGC_BROLL_ENABLED — with sub-flags for each phase tier. Production reference-first runs stayed on Phase 2 by default. Internal testers and script-first beta users got Phase 3.

function resolveBrollPhase(flags: FeatureFlags, run: RunConfig): BrollPhase {
  if (!flags.UGC_BROLL_ENABLED) return 'disabled';
  if (run.mode === 'script-first' && flags.UGC_BROLL_PHASE_3) return 'ai-generated';
  if (run.referenceVideoUrl && flags.UGC_BROLL_PHASE_2) return 'reference-swipe';
  if (flags.UGC_BROLL_PHASE_1) return 'manual';
  return 'disabled';
}

Gating also controlled UI surface area. Phase 1 exposed beat markers in the planner. Phase 2 added a "swipe timing from reference" button. Phase 3 added cutaway preview tiles, editor brief fields, and insert-between-scenes handles. Shipping all three UI surfaces at once would have overwhelmed editors who only needed manual markers.

Insert-between-scenes control

A late addition to the gating PR: b-roll beats are not only mid-scene offsets. Editors can insert cutaways between scene boundaries — useful when the script has a natural paragraph break that does not map to a mid-scene timestamp. The planner renders these as bridge segments in the stitch timeline, with J-cut audio overlap from the adjacent A-roll.

End-to-end fixes in Phase 3

Phase 3 was not just generation — it required stitch-time and VO pipeline changes:

• Cloned-voice VO on cutaways — supplementary lines use the same ElevenLabs voice clone as the presenter, with shorter pacing targets
• Stitch ordering — b-roll segments interleave with A-roll clips using explicit timeline indices, not filename sort order
• Duration sync — cutaway length clamped to beat duration; overflow triggers trim, underflow triggers hold-frame extension
• E2E test coverage — full run with b-roll beats through stitch export validated in CI

Phase comparison metrics

What I'd do differently

I would define the people-free constraint in the shared prompt library from Phase 1, not Phase 3. We retrofitted it after identity confusion reports, but manual-phase editors were already inserting placeholder slots that Phase 3 later filled with face-containing clips.

I would also expose phase tier in the run manifest for debugging. Support tickets saying "my b-roll looks wrong" required log diving to discover whether the run used manual beats, reference swipe, or AI generation.

The pattern: phased capability with shared data model

The three-phase b-roll arc is a reusable template for complex generative features:

1. Manual control — prove the data model and editor mental model
2. Automated inference — reduce editor labor when structured input exists
3. Full generation — AI produces the artifact, grounded by vision and constrained by domain rules

Each phase reuses the same BrollBeat structure. Feature flags gate progression. Domain constraints — people-free b-roll, product lock, captioned link validation — apply regardless of phase. That separation let us ship Phase 1 to all editors while Phase 3 baked in beta, without forking the stitch pipeline.