Crop or Pad? Designing Fixed Aspect Ratios for Batch Image Conversion

Convertim, the image converter I build, can already turn a 4:3 photo into a 1080×1350 image. It does it by stretching the photo until it fits.

That's its Force dimensions mode: exact width, exact height, aspect ratio ignored. Under the hood it's sharp's fit: 'fill'. It does exactly what it says, and for a photo it's the worst of the three possible answers. The other two are what I'm working on now: cropping to fixed ratios, and padding to them.

Convertim is a portable Windows image converter: a desktop app and a CLI on top of the same engine, built on sharp (libvips), running entirely on your machine. Its whole point is batches. Drop two hundred files, pick a format, go. That single fact changes how crop has to be designed, and almost none of the hard decisions turned out to be technical.

I'm writing this while building it. Most of it is decided now, and I'll say why. What isn't settled yet is whether those decisions survive a few hundred real images.

Fixed ratios, not a crop handle

The first decision was what not to build: a freeform crop, with a rectangle you drag over each image.

Freeform crop is one decision per image. That's perfect for one photo, and it's what a photo editor is for. Convertim is built for two hundred at a time, and two hundred drag handles is not a feature, it's an afternoon.

So crop in Convertim means fixed aspect ratios, chosen once for the whole batch: eleven common ones, from 1:1, 4:5 and 9:16 to 16:9 and 3:1, plus a custom one anywhere between 1:10 and 10:1. One rule, applied to everything. Presets by destination come on top: pick where the image is going (an Instagram portrait, a story, a YouTube thumbnail, a LinkedIn banner) and get the exact size and ratio. They're built on the same rule.

Every ratio change costs the same

When an image doesn't match the target ratio, there are only two honest answers:

• Crop what sticks out. The frame is full, and some pixels are gone.
• Pad what's missing. Every pixel survives, and part of the frame is filler.

What I didn't expect is that they cost exactly the same. Turning a 4000×3000 photo (4:3) into 4:5, at its own resolution:

It isn't a coincidence. The fraction you lose by cropping is always the fraction you fill by padding: one minus the smaller aspect ratio divided by the larger. A 16:9 video frame turned into a 9:16 story costs 68%, whichever way you do it.

So the question is never which one is cheaper. It's whether you would rather pay in lost pixels or in filler, and that depends on the image, which is exactly what a batch doesn't let you look at.

Where to anchor the crop

Crop has a second question hiding inside it: which part to keep. sharp's cover takes a position, and the answers come in three families:

// Same image, same 4:5 box, three different results.
sharp(input).resize({ width: 1080, height: 1350, fit: 'cover', position: 'centre' })
sharp(input).resize({ width: 1080, height: 1350, fit: 'cover', position: 'north' })
sharp(input).resize({ width: 1080, height: 1350, fit: 'cover', position: sharp.strategy.attention })

• Centre is the predictable one. It's also the one that cuts the top of someone's head when a portrait is cropped to a wider ratio, because in most portraits the face sits in the upper half of the frame.
• Top (north) protects the heads. It only does anything when the crop removes height; when it removes width, it behaves exactly like centre. And it fails on anything whose subject sits low in the frame.
• Content-aware strategies look at the pixels. entropy keeps the busiest region; attention looks for luminance frequency, colour saturation and skin tones. On average they often do better than either of the others.

That "on average" is the problem. In a batch, the anchor doesn't need to be clever, it needs to be predictable. With centre or top, someone can picture what will happen to the whole batch before pressing the button. With attention, the only way to know where image 137 was cropped is to open image 137, and saving you from opening every image is the reason a batch tool exists.

So the default is centre, and the anchor is a 3×3 grid: the centre, the four edges and the four corners. Content-aware crop is there too, as Smart crop, but it's a checkbox that starts unticked, with a short note right next to it. It works well with one clear subject on a simple background, with a subject that has more colour or contrast than its surroundings, and with portraits, because it weighs skin tones. It can fail with busy backgrounds like foliage or crowds, with several subjects spread across the image, and with a dull subject on a louder background. It doesn't recognise faces or objects: it follows colour, contrast and skin tones. Ticking it is a choice to trade predictability for a better average, and the note is there so that it's an informed one.

What to pad with

Padding has its own question: what goes in the gap?

For formats with an alpha channel (PNG, WebP, AVIF), transparency looks like the obvious answer. The image stays exactly what it was, and the frame around it is nothing. It's one of the options, but not the default: the default is a solid colour, white, for every format, so the same batch gets the same frame whether it comes out as PNG or as JPG.

JPG has no alpha channel, and this is where sharp surprised me. Pad with a transparent background, save as JPG, and the padding comes out black. Not the colour you asked for: pad with transparent white and you still get black, because the colour of a fully transparent pixel doesn't survive the trip.

The obvious fix doesn't work either:

// Looks right, and isn't: the padding is still black in the JPG.
await sharp(input)
  .resize({ width: 1080, height: 1350, fit: 'contain', background: { r: 0, g: 0, b: 0, alpha: 0 } })
  .flatten({ background: '#ffffff' })
  .jpeg()
  .toFile(output)

sharp flattens the alpha before it resizes, whatever order the calls are written in, so flatten never sees the padding that contain adds afterwards. It does flatten the transparency that was already in the input; it just can't reach the gap. What works is deciding the colour before the padding exists:

// Without an alpha channel, the gap needs a solid colour from the start.
const background = supportsAlpha(format)
  ? { r: 0, g: 0, b: 0, alpha: 0 }
  : '#ffffff'

await sharp(input)
  .resize({ width: 1080, height: 1350, fit: 'contain', background })
  .toFormat(format)
  .toFile(output)

With that in place, transparent padding saved as JPG comes out white, not black, and a colour pad is the background of the whole frame: the image's own transparency is flattened onto that colour too, not only the bands around it.

And it isn't only a padding question: a transparent PNG converted to JPG faces the same choice, crop or no crop. In Convertim 2.0 it came out on black. In the update it's flattened onto white.

Besides a colour and transparency, there's a third option: blur. The gap is filled with the image itself, scaled up to cover the whole frame and heavily blurred, with the full image on top. It's the usual way to turn a landscape photo into a 9:16 story without colour bars. The blur runs on a copy at an eighth of the final size, which is then scaled back up: it looks like a huge blur at full size and costs a fraction of one.

The order of operations

Then there's where crop goes in the pipeline. In Convertim 2.0 the pipeline was written in this order: the user's rotation and flips, colour adjustments, resize, watermark, encode. Crop changes what has to come before what:

1. Orientation. A phone photo stored sideways, with an EXIF tag saying which way is up, has to be turned upright before anything measures it. Skip that, and the app measures a portrait as a landscape and crops the wrong side. (Testing this turned up something I'll write about next week.)
2. The user's rotation. Crop to 4:5 and then rotate 90°, and you've made a 5:4.
3. Crop or pad, and resize. In sharp they're one operation: cover scales until the box is full and cuts the rest, contain scales until the image fits and fills the rest.
4. Watermark. After the crop, never before. A watermark in a corner is exactly what a crop removes first, and its size is computed from the final dimensions, so it has to wait for them anyway.
5. Encode, with the padding colour already decided.

That's the order it runs in now, in a single pipeline shared by conversion, optimisation and the preview, so the preview shows exactly what will be written.

The trap is that a sharp chain isn't a script. Some operations run in a fixed internal order, whatever order you call them in. In the version Convertim uses (0.34), a 90° rotate() runs before the resize even if you chain it after, and so does flatten(), which is the JPG trap above. extract(), sharp's manual crop, is the exception: it runs where you put it.

So the order of operations isn't something I trust from reading the code. It's something to test, by checking the output dimensions and where the pixels ended up. More on that next week too.

Small images

One more decision that looks technical and isn't: what happens when the input is smaller than the destination? A 640×480 photo going into a 1080×1350 preset.

sharp enlarges by default, so cover scales it up and crops, and you get a soft 1080×1350. Convertim's existing quality presets already behave like that: the Web preset takes a 400-pixel-wide image up to 1200, and there's a test checking that it does.

The alternative, withoutEnlargement, has a surprise when combined with cover: a small image comes back untouched. Not cropped, not padded, just its original size in its original ratio. So "never upscale" and "always 4:5" can't both win. If a preset promises a ratio, either the image gets enlarged, or the crop happens at the image's own resolution and the preset only ever shrinks.

Convertim does both, depending on what you ask for. A ratio without a final size scales nothing: crop keeps the largest box of that ratio inside the image, and pad builds the smallest canvas that holds it (up to 16,383 pixels a side, WebP's limit). A final size, or a social preset, which always has one, is a promise of exact pixels, so the image is scaled to it, up if it has to be, and the preview says when an image is being enlarged.

The default is the product

On a single image, a bad crop costs two seconds: you see it, you undo it, you move on. In a batch, nobody sees it. Two hundred images go in, two hundred come out, and the first time anyone looks closely at number 137 might be after it's been posted.

That's why the default deserves more time than the crop itself. The crop is a handful of sharp options. The default is a decision applied two hundred times without anyone looking, and it's what most people will get, because most people never change a default.

Here's where things stand:

• Decided: fixed ratios, no freeform crop, one rule per batch. Orientation and rotation before the crop, the watermark after it. A solid padding colour whenever the format has no alpha channel.
• The defaults: crop, anchored in the centre, with smart crop as an opt-in. White padding for every format, with transparency and blur as options. No scaling without a final size; with one, the image is enlarged if it has to be, and the preview says so.
• Still to prove: that those defaults hold up on a few hundred real images.

Every one of those defaults went through the same three questions. Which failure is easier to spot in a grid of two hundred thumbnails? Which failure leaves the whole image in the output, and which one doesn't? And can someone predict the result for the entire batch before pressing the button? Centre and white are the answers anyone can predict before pressing it. But a question answered in the abstract isn't answered yet. These are the defaults going into testing, not a promise: if a few hundred real images say otherwise, they change.

In a batch tool, the default isn't a detail of the feature. It is the feature, for everyone who drops the files, presses the button and never opens the options. That's why this part gets more time than the crop itself, and why crop doesn't ship until the defaults have been argued with real images. No date yet.

Next week: the test images I use to break this before anyone else does.