Seam-Free Video Splicing: Grade-Match, End-Frame Interpolation, and Karaoke Captions

Splicing AI-generated sections into existing video is easy. Making the splice invisible is not. When Variant Multiplier swaps a body section or Hook Machine stitches a generated opener onto A-roll, the viewer's eye catches the cut — a colour temperature shift, a brightness jump, a frame where motion stutters. This post covers the three techniques that repair those seams: deterministic grade-match, end-frame interpolation, and word-level karaoke captions.

Neural colour grading is impressive in demos. Deterministic histogram matching is what you ship when you need the same splice to look identical on the thousandth run.

The visual seam problem

AI video engines render with their own implicit colour pipeline — white balance, contrast curve, saturation profile. Source footage from a real UGC shoot carries the phone camera's pipeline. Concatenating them with FFmpeg's concat demuxer produces a visible discontinuity at every boundary:

• Colour shift — generated section is cooler/warmer than source
• Exposure jump — generated section is brighter/darker
• Motion discontinuity — subject position does not match at cut-back
• Caption desync — text timing does not align with new VO

Grade-match: deterministic colour transfer

feat/grade-match implements histogram-based colour transfer from a reference frame (the last frame of the preceding source section) to the first and last frames of the generated section. The algorithm:

1. Extract reference frame RGB histogram from source video at splice point
2. Extract target frame histogram from generated section boundary
3. Compute per-channel lookup table mapping target → reference distribution
4. Apply LUT to all frames in the generated section

interface ChannelLUT {
  r: Uint8Array;  // 256-entry lookup
  g: Uint8Array;
  b: Uint8Array;
}

function computeGradeMatchLUT(
  referenceFrame: RGBBuffer,
  targetFrame: RGBBuffer,
): ChannelLUT {
  const refHist = { r: histogram(referenceFrame.r), g: histogram(referenceFrame.g), b: histogram(referenceFrame.b) };
  const tgtHist = { r: histogram(targetFrame.r), g: histogram(targetFrame.g), b: histogram(targetFrame.b) };

  return {
    r: matchHistograms(tgtHist.r, refHist.r),
    g: matchHistograms(tgtHist.g, refHist.g),
    b: matchHistograms(tgtHist.b, refHist.b),
  };
}

function applyLUT(frame: RGBBuffer, lut: ChannelLUT): RGBBuffer {
  return {
    r: frame.r.map(v => lut.r[v]),
    g: frame.g.map(v => lut.g[v]),
    b: frame.b.map(v => lut.b[v]),
  };
}

The FFmpeg integration wraps this as a filter graph applied before concat:

# Grade-match filter applied to generated section before concat
ffmpeg -i generated_section.mp4 -i reference_frame.png \
  -filter_complex "[0:v]lut3d=grade_match.cube[graded];[graded][1:v]..." \
  -c:v libx264 -crf 18 graded_section.mp4

Why deterministic beats neural for colour transfer

Neural colour grading excels when you need creative grading — "make this look like golden hour." Splice repair needs matching, not creativity. Histogram transfer solves matching without introducing artefacts.

End-frame interpolation

Grade-match fixes colour. It does not fix motion. When a generated section ends and the stitch cuts back to original source footage, the subject's head position, hand gesture, and background motion rarely align. feat/end-frame generates a transition frame that bridges the gap.

The pipeline:

1. Extract last frame of generated section (after grade-match)
2. Extract first frame of following source section
3. Generate interpolated frame(s) blending pose/lighting between the two
4. Insert 2–4 interpolated frames (80–160ms at 25fps) before the hard cut-back

interface EndFrameConfig {
  interpolationFrames: number;  // typically 2–4
  method: 'optical-flow' | 'cross-dissolve' | 'model-interp';
}

async function buildEndFrameTransition(
  generatedLastFrame: Buffer,
  sourceFirstFrame: Buffer,
  config: EndFrameConfig,
): Promise<Buffer[]> {
  if (config.method === 'cross-dissolve') {
    return linearCrossDissolve(generatedLastFrame, sourceFirstFrame, config.interpolationFrames);
  }
  // optical-flow or model-based for higher quality when motion delta is large
  return opticalFlowInterp(generatedLastFrame, sourceFirstFrame, config.interpolationFrames);
}

Cross-dissolve handles small motion deltas (talking head, minimal gesture change). Optical-flow interpolation handles larger deltas but costs more compute. The stitch pipeline selects method based on motion magnitude between boundary frames.

Cut-back smoothness metrics

Word-level transcript selection

Seam repair is visual, but splice accuracy starts with temporal precision. feat/word-level-selection provides karaoke-style word boundaries that define exactly where the generated section begins and ends in the source timeline.

interface WordSpan {
  word: string;
  startMs: number;
  endMs: number;
  confidence: number;
}

function buildSpliceTimeline(
  words: WordSpan[],
  sectionStart: number,
  sectionEnd: number,
  generatedDurationMs: number,
): StitchTimeline {
  const preRoll = words.slice(0, sectionStart);
  const postRoll = words.slice(sectionEnd + 1);
  return {
    segments: [
      { type: 'source', wordSpan: preRoll },
      { type: 'generated', durationMs: generatedDurationMs },
      { type: 'source', wordSpan: postRoll },
    ],
    splicePoints: [
      { ms: preRoll.at(-1)?.endMs ?? 0, repair: ['grade-match', 'end-frame'] },
      { ms: preRoll.at(-1)?.endMs ?? 0 + generatedDurationMs, repair: ['grade-match', 'end-frame'] },
    ],
  };
}

Word-level boundaries prevent the most common splice error: cutting mid-phoneme. Silence-based selection gets close; word selection gets exact.

Karaoke captions: libass with drawtext fallback

After splicing, captions must reflect the new timeline — including regenerated VO in the swapped section. feat/karaoke-captions renders word-highlighted captions synchronized to the final stitched audio.

libass rendering (primary path)

libass renders ASS subtitles with per-word highlight timing — the "karaoke" effect where each word lights up as spoken. This requires font files in the container and FFmpeg compiled with libass support.

function generateAssKaraoke(words: WordSpan[], style: CaptionStyle): string {
  const lines = ['[Script Info]', 'Title: Karaoke', '', '[V4+ Styles]', `Style: Default,${style.font},${style.fontSize},...`, '', '[Events]', 'Format: Layer, Start, End, Style, Text'];
  for (const word of words) {
    const start = msToAssTime(word.startMs);
    const end = msToAssTime(word.endMs);
    lines.push(`Dialogue: 0,${start},${end},Default,{\\c&H${style.highlight}&}${word.word}`);
  }
  return lines.join('\n');
}

drawtext fallback

When libass is unavailable (minimal Docker images, missing fonts), the pipeline falls back to FFmpeg's drawtext filter — one word at a time, positioned bottom-center:

ffmpeg -i stitched.mp4 -vf "
  drawtext=text='{{word}}':fontfile=/fonts/Inter-Bold.ttf:
  fontsize=48:fontcolor=white:borderw=2:bordercolor=black:
  x=(w-text_w)/2:y=h-120:enable='between(t,{{start}},{{end}})'
" -c:v libx264 -c:a copy captioned.mp4

drawtext lacks the smooth highlight animation of libass but guarantees captions render in any environment. The fallback is automatic — runtime detection, no config flag.

Composing the splice repair stack

In production, all three techniques compose in a fixed order:

1. Word-level selection — define precise splice boundaries
2. Grade-match — colour transfer on generated section
3. End-frame interpolation — motion bridge at cut-back points
4. Concat — FFmpeg stitch with audio crossfade
5. Karaoke captions — render on final stitched output

async function spliceWithRepair(
  source: VideoAsset,
  generated: VideoAsset,
  section: SectionBounds,
  words: WordSpan[],
): Promise<StitchedOutput> {
  const refFrame = await extractFrame(source, section.startMs - 1);
  const graded = await gradeMatch(generated, refFrame);
  const endFrames = await buildEndFrameTransition(
    await extractFrame(graded, graded.durationMs - 1),
    await extractFrame(source, section.endMs),
  );
  const stitched = await concatSegments([source.before(section), graded, ...endFrames, source.after(section)]);
  const captioned = await renderKaraokeCaptions(stitched, realignWords(words, section));
  return captioned;
}

What I'd do differently

I would compute grade-match LUTs once per source video and cache them keyed by source frame hash. Currently each section swap recomputes histograms from scratch — cheap individually, but redundant in batch queue runs on the same source.

I would also expose seam quality as a numeric score in the UI, not just pass/fail. Editors could choose to accept a 12% seam risk to save 800ms of optical-flow interpolation when shipping variants quickly.

The pattern: repair at the splice, not in the generator

Video generation models will get better at colour consistency over time. They will not solve splice repair — the source footage's colour pipeline is external to any model. Building deterministic repair as a post-generation stitch step decouples splice quality from model quality and keeps the stack reproducible at production scale.