BiSeNet or XSeg? Choosing the Right Face Mask for an Image Pipeline

A face parser can return a map of facial parts or a mask of the face as a whole. Those outputs can look similar in a visualization, but they support different operations.

If an editor needs to select hair or lips, it needs region labels. If a compositing step needs a face-shaped selection, a single mask may be sufficient. Choosing between BiSeNet and XSeg starts with that distinction, before comparing model size or inference time.

This article adapts my original BiSeNet/XSeg comparison into an implementation guide for UniFace. It retains the output examples while separating API behavior from observations on a small set of portraits.

Two outputs with different meanings

BiSeNet's 19 classes include background, skin, eyes, eyebrows, nose, lips, and hair. A selection is a comparison against one or more class IDs. XSeg supplies one mask; it cannot tell the application which selected pixels are lips or skin.

In this example, XSeg's mask excludes the hair. It should not be treated as a full portrait or head mask. If the task is separating a person, including hair, from a background, portrait matting is a separate option.

Coordinate space is part of the interface

A mask is only useful if it lines up with the pixels being edited. BiSeNet returns labels in the coordinate system of its input crop. XSeg's normal parsing call aligns the face using landmarks, runs inference, then transforms the result back into the original image.

The following example makes those two coordinate systems explicit. Install UniFace with the CPU runtime using pip install "uniface[cpu]" and supply a portrait containing one face.

import cv2
import numpy as np
from uniface import BiSeNet, RetinaFace, XSeg

image = cv2.imread('portrait.jpg')
if image is None:
    raise FileNotFoundError('Could not read portrait.jpg')

faces = RetinaFace().detect(image)
if len(faces) != 1:
    raise ValueError(f'Expected one face, found {len(faces)}')
face = faces[0]

# Expand the detector box to retain more hair and surrounding context.
x1, y1, x2, y2 = face.bbox[:4]
bw, bh = x2 - x1, y2 - y1
height, width = image.shape[:2]
left = max(0, int(np.floor(x1 - 0.2 * bw)))
top = max(0, int(np.floor(y1 - 0.4 * bh)))
right = min(width, int(np.ceil(x2 + 0.2 * bw)))
bottom = min(height, int(np.ceil(y2 + 0.2 * bh)))
crop = image[top:bottom, left:right]

labels = BiSeNet().parse(crop)
hair_in_crop = (labels == 17).astype(np.uint8) * 255
cv2.imwrite('hair_crop.png', hair_in_crop)

face_mask = XSeg().parse(image, landmarks=face.landmarks)
cv2.imwrite('face_mask.png', (face_mask * 255).astype(np.uint8))

The crop margins are a preprocessing choice, not a guarantee that every hairstyle fits. The saved hair mask belongs to the expanded crop. To position it in the full frame, use the recorded left and top offsets. The XSeg mask already has the full image dimensions.

UniFace also exposes XSeg.parse_aligned() for callers that already have an aligned crop. That path returns a mask in crop coordinates and does not perform the full-image transformation.

What the example portraits can establish

The original comparison examined three faces. In that small sample, the larger BiSeNet backbone did not consistently preserve more detail, and XSeg produced an almost binary mask despite its floating-point output.

These are observations about those examples. They do not establish a general backbone ranking or a minimum usable face size. Counting which classes appear also does not measure whether the boundaries are correct; that requires reference annotations and a segmentation metric.

This occluded portrait illustrates another evaluation question: does the mask include the obstruction? The example is worth inspecting, but one scarf image does not establish performance on hands, glasses, or other occlusions.

Match the evaluation to the edit

For region-specific editing, evaluate the classes the editor actually uses and inspect boundaries at the intended image resolution. For face compositing, examine mask coverage, alignment errors, and leakage into hair or foreground objects.

A value between zero and one is not evidence of physically accurate transparency or a calibrated probability. Smoothing a face mask can soften an edge; it does not recover hair detail or turn the output into a portrait alpha matte.

BiSeNet and XSeg provide different selections. The useful comparison is whether each output supports the next image operation with acceptable errors on representative inputs.

See the parsing API documentation, the BiSeNet implementation and weights, and the XSeg ONNX repository for implementation details. Check the model licence terms before deployment.