This week two of our own help pages contradicted each other. MagicRemover's passersby page promised that only the area you brush changes; the people page said the whole image may shift slightly. Only one could be true, so I read our pipeline end to end instead of trusting either page.
The people page was right. An instruction-based eraser doesn't fill a hole: it re-renders the entire photo from a text instruction, and your brush stroke only decides what that instruction says. This post walks through how that pipeline works in production, why it was worth switching to, and what it costs the pixels you never touched.
Disclosure: I build MagicRemover, the browser-based AI eraser this post is about. The draft was written with AI assistance and every code excerpt and number was checked against the repository and production logs.
Two ways to build an eraser
There are two broad designs, and they fail in opposite ways.
Mask inpainting takes the image plus a binary mask. The model generates pixels only inside the mask, and the pipeline copies everything outside it straight from the original. Outside the brush, the result is pixel-identical to your upload. The weakness is context: if you brush a person but not their shadow, the shadow stays, because the model was only allowed to touch the mask.
Instruction editing takes the image plus a sentence such as "remove the man in the blue shirt". The model returns a complete new image. Because it understands the scene, it can remove the shadow and the reflection you didn't brush. The cost is that every pixel in the output was generated, including the ones you wanted left alone.
MagicRemover's photo tools ran on mask inpainting until May 2026. In early May we added a labelling step; in June we replaced the backend with an instruction-following image editor that takes only an image and a prompt. Fills got noticeably more coherent, especially around shadows. What we didn't do was update every sentence on the site that still described the old behaviour.
How a brush stroke becomes a sentence
Here is the path a removal takes today.
1. The browser shrinks the photo to about 1 megapixel
Every upload goes through one helper before anything leaves the browser:
/** Cap uploads at ~1 megapixel (matches the model's ~1MP working resolution). */
export const MAX_PIXELS = 1_000_000;
/** Secondary edge cap for very wide panoramas. */
export const MAX_EDGE = 2048;
export function fitTarget(w: number, h: number) {
let scale = 1;
const pixels = w * h;
if (pixels > MAX_PIXELS) scale = Math.sqrt(MAX_PIXELS / pixels);
const longest = Math.max(w, h) * scale;
if (longest > MAX_EDGE) scale *= MAX_EDGE / longest;
return {
width: Math.max(1, Math.round(w * scale)),
height: Math.max(1, Math.round(h * scale)),
};
}
A 4000×3000 phone photo becomes 1155×866. There is no point sending more: the editing model works at roughly one megapixel and returns an image at that size, so extra pixels would only make the request slower and bigger.
2. The browser sends two copies
The editor posts the clean, downscaled original and a second copy with the user's brush strokes filled solid red. The red copy is never edited; it exists so the next step can see what was marked.
3. A vision model names what is under the red
A vision-language model receives both images and a tool-specific instruction. The people tool's instruction starts like this:
Identify the person or people inside the red-marked region of image 2.
Your reply MUST uniquely identify ONLY the red-marked subjects so they can
be deleted without affecting any similar-looking person ...
It answers with a short label, along the lines of "the man in the blue shirt on the left". In the nine removals logged since our latest deploy, all nine got a label, and the labelling step took about 2.5 seconds on average. That's a small sample, but it matches what we see day to day.
4. The editor gets the clean photo and a sentence
The label is dropped into a per-tool template. The people tool uses:
Completely remove {label} from the photo, leaving no trace, shadow, or
outline, and realistically reconstruct the background and scene behind
them. Keep everyone and everything else exactly unchanged.
The editor receives the clean original, never the red copy, so there's no risk of red pixels leaking into the result.
5. If labelling fails, fall back to the red copy
When the vision model can't produce a label, the editor gets the red-marked copy and a generic instruction instead:
Completely remove whatever the red area covers, then remove the red marking
itself and fill the space with background that naturally matches the
surroundings. Leave no trace, shadow, or outline. Keep everything outside
the red area exactly unchanged.
That keeps the tool working when the labeller is down, at the cost of slightly worse results, because "the red area" is a vaguer target than "the man in the blue shirt".
Why the whole photo changes
Notice the last sentence of both prompts: "Keep everything else exactly unchanged." It's a request, not a guarantee. The model regenerates the full frame, and nothing in the pipeline copies the original pixels back.
In practice the untouched parts of the photo look the same, but they aren't bit-identical. The things most likely to drift are fine repeating textures (fabric, foliage, gravel), small text away from the brushed area, and faces in the background. Colour can shift slightly across the whole image too.
That's acceptable for the use most people have: a holiday photo going to Instagram or a chat. It's not acceptable for an e-commerce product shot that has to match its colour profile, or a scanned document where every other character must survive untouched. So the pages now say it plainly: changes concentrate in the brushed area, the rest looks the same but isn't pixel-identical.
Why we don't paste the original back (yet)
The obvious fix is to composite: keep the edited pixels inside the brushed area, take the original's pixels everywhere else. It's about twenty lines with sharp. This is an example, not production code; it's tested with sharp 0.34.5 on Node 24:
import sharp from "sharp";
const { width, height } = await sharp("original.png").metadata();
// 1. Grow the brushed area so the seam lands outside the object's edge:
// blur the mask, then turn every non-black pixel white.
const blurred = await sharp("mask.png").resize(width, height).blur(8).toBuffer();
const grown = await sharp(blurred).threshold(1).toBuffer();
// 2. Feather the grown edge and keep a single raw channel to use as alpha.
const alpha = await sharp(grown).blur(6).extractChannel(0).raw().toBuffer();
// 3. The edited image at the original's size, with that alpha channel.
const rgb = await sharp("edited.png").resize(width, height).removeAlpha().toBuffer();
const patch = await sharp(rgb)
.joinChannel(alpha, { raw: { width, height, channels: 1 } })
.png()
.toBuffer();
// 4. Lay the patch over the untouched original.
await sharp("original.png").composite([{ input: patch }]).toFile("result.png");
Two gotchas cost me time while testing it. sharp applies operations in a fixed internal order, not in the order you chain them, so blur, threshold and blur again only behave as written when each runs in its own pipeline. And removeAlpha() chained after joinChannel() quietly strips the channel you just joined, which is why step 3 builds the RGB buffer first.
So why isn't this in production? Because it gives back what instruction editing bought us.
- Shadows come back. The editor removes a person's shadow even if the user brushed only the person. Composite outside the mask and the shadow is pasted right back from the original. Growing the mask helps a little; it can't know where a long evening shadow ends.
- Seams appear. The editor relights the area it rebuilt. Where its version meets the original, a feathered edge turns a sharp seam into a soft halo, but on a flat sky or a white wall you can still see the step in brightness.
- Resolution stays the same. Compositing keeps the original pixels outside the mask, but if you composite onto the downscaled original, the output is still about one megapixel. To get full resolution back you'd upscale the patch onto the full-size upload, and a 1 MP patch dropped into a 12 MP photo looks soft next to its neighbours.
A hybrid is on our list: composite only when the brushed area is small and far from strong shadows, and leave the full regeneration otherwise. Until then, the honest move was to change the copy rather than the pipeline.
When an instruction editor is the wrong tool
- You need everything outside the mask to stay bit-identical: product photos with strict colour, documents, anything used as a record. Use mask inpainting with compositing, or Photoshop's Remove Tool on the full-resolution file.
- You need print resolution. Any eraser that works at about one megapixel, MagicRemover included, will hand you back a smaller image.
- The image is dense with text you want to keep. Regenerating the frame can subtly alter characters far from the brush.
Where MagicRemover fits
MagicRemover is a browser-based AI eraser: you upload a photo, brush over a person, object, text, sticker or watermark, and it returns the photo with that thing removed, usually in well under a minute. It's built for photos headed to the web and social media, with 2 free removals a day without an account and 2 more after Google sign-in. Don't use it when you need full resolution or pixel-exact output outside the brushed area; for those jobs a desktop editor on the original file is the right call.
The tool walkthroughs, including the people remover discussed above, are at magicremover.org.
If you've shipped an eraser yourself: do you composite the original back, and how do you deal with shadows that fall outside the user's mask?
