Why the same face doesn't score 1.0 – lessons from building a face-similarity search in Python

I built FaceMatch as a portfolio project: you give it a photo, pick a face, and it ranks the followers or following of an Instagram profile by how similar their profile pictures (and optionally their recent posts) look to that face.

Before anything else: a similarity score is not proof of identity. The tool ranks visual similarity, nothing more. I'll come back to the privacy side at the end.

Repo: github.com/Eight7Six/insta-facematch

How it works

1. Detect all faces in the target image (RetinaFace, with MTCNN and OpenCV as fallbacks) and let the user click the one to search for.
2. Compute a 512-dimensional embedding for that face with DeepFace (Facenet512).
3. Log in via instagrapi and load the followers/following of one or more profiles.
4. For every account, download the profile picture and optionally the last N posts, compute embeddings for every face found, and keep the best cosine similarity.
5. Print a ranking, a report of accounts that could not be compared (with a reason), and export both to CSV.

The comparison itself is tiny:

def cosine_similarity(a: np.ndarray, b: np.ndarray) -> float:
    a = a / (np.linalg.norm(a) + 1e-9)
    b = b / (np.linalg.norm(b) + 1e-9)
    return float(np.dot(a, b))

Everything around it is where the interesting problems were.

Lesson 1: "No matching distribution" usually means "wrong Python"

When I tested a clean install on a second machine, pip install -r requirements.txt failed with:

ERROR: Could not find a version that satisfies the requirement tensorflow==2.21.0 (from versions: 2.22.0rc0)

The hint is in the parentheses: pip saw exactly one TensorFlow version. TensorFlow ships pre-built wheels per Python version, and 2.21.0 only has them for Python 3.10–3.13. The second machine had the latest Python, 3.14, and the only release with 3.14 wheels was a release candidate.

The fix was not in the code but in the docs: the README now states the supported Python versions up front and tells users to create the virtual environment with an explicit version (py -3.13 -m venv .venv) instead of a plain python -m venv, which silently picks whatever the default is.

Lesson 2: The same person doesn't score anywhere near 1.0

My test target was a public, verified account. I expected a score close to 1. I got around 0.5.

That's normal. A score of 1.0 only happens for identical images. For Facenet512, DeepFace's own verify() threshold is a cosine distance of 0.30, i.e. a similarity of 0.70, and that is tuned on good, frontal image pairs. In my case several things pulled the score down:

• The target face was tiny. My target was a screenshot about 480 px wide; the face was maybe 40–50 px. Facenet512 works on 160×160 inputs, so the face got upscaled three to four times.
• Pose, expression and lighting. A serious portrait compares badly to a broad smile under stadium lights.
• Only one post was actually compared – more on that in Lesson 4.

I kept the default threshold at 0.55 on purpose: it is below DeepFace's 0.70 and acts as a generous pre-filter for manual review, not as a decision.

There's also a statistical trap: the tool keeps the best score per account across all faces in all images. With hundreds of accounts and several images each, you make thousands of comparisons, and some stranger will land above the threshold by chance. More images do not automatically mean better results.

Lesson 3: Follower lists only give you thumbnails

The profile picture URLs in follower lists point to 150×150 px images – you can see s150x150 in the URL. My first idea was to edit that part of the URL. That doesn't work: Instagram's CDN URLs are signed, and a modified URL is rejected.

The full-size picture needs one extra profile request per account:

details = client.user_info(user["id"])
if details.profile_pic_url_hd:
    user["profile_pic_url"] = str(details.profile_pic_url_hd)
user["is_private"] = details.is_private

Two side effects: the run gets a bit slower (one more request per account), and the "private" flag becomes reliable, because the follower list often doesn't populate it at all.

Lesson 4: Skipping carousels skipped the best match

Originally the code only evaluated single-photo posts and skipped everything else. With "3 posts per account", my test account's last three posts were a reel, a photo and a carousel – so only one post was compared.

After adding carousels (using only the first photo of each), the score for the same account went from about 0.5 to 0.60. That first carousel photo came from the same match as the target image: same jersey, same lighting, same broad smile.

I deliberately use only the first photo. A carousel can hold up to 20 images; evaluating all of them would multiply the runtime and – see Lesson 2 – the number of chance matches. Reels stay skipped: their cover frames are often blurry or covered with text.

Lesson 5: Detect once, or your indices drift apart

The log showed this on every run:

Detector retinaface: 7 face(s) found in Target.png
Detector retinaface: 6 face(s) found.

Same image, same detector, different counts. The program detected faces twice: once with DeepFace.represent(align=True) for the embeddings, and once with DeepFace.extract_faces(align=False) to draw the boxes in the selection window. With align=True, DeepFace pads the image with a black border before detection, so the detector sees a slightly different picture, and borderline faces flip in or out.

A click was mapped back to the right embedding via bounding-box overlap, so the result was correct. But the numbers in the window referred to a list of six faces, while the --person-index CLI option referred to a list of seven. The second detection also cost about ten seconds.

The fix: detect once and pass those faces to the picker. One list, one numbering, and the overlap-matching code could be deleted.

Lesson 6: A continue that skipped the cleanup

Downloaded images are supposed to be deleted right after analysis. Images without a detectable face stayed on disk anyway:

faces, _ = get_embeddings(img_path, DETECTOR_ORDER)
if not faces:
    continue                      # jumps past the cleanup below
...
img_path.unlink(missing_ok=True)

try/finally fixes it and also covers exceptions during analysis:

try:
    faces, _ = get_embeddings(img_path, DETECTOR_ORDER)
finally:
    img_path.unlink(missing_ok=True)
if not faces:
    continue

I had first documented this as a "known issue". Fixing it took three lines – a good reminder that a known issue in a README is sometimes just a bug you haven't fixed yet.

Privacy and responsible use

This is the part I thought about most:

• Similarity is not identity. Every "possible match" is an unverified hypothesis.
• Face embeddings are biometric data. Under the GDPR and similar laws, processing them for third parties generally needs a legal basis. Running a tool like this against real people without their consent may be unlawful.
• Instagram's Terms of Use restrict automated access. instagrapi uses Instagram's undocumented mobile API, so use a throwaway account you are prepared to lose.
• The demo only uses public figures. For testing, I created a separate account that follows ten verified public accounts.

The README states all of this up front, and the repository ignores session files, target images, downloaded pictures and result CSVs by default.

Wrapping up

The actual face comparison is three lines of NumPy. The real work was everything around it: packaging and Python versions, data quality, API quirks, consistent state between GUI and CLI, cleanup, and documentation that's honest about the limits.

Feedback on the code – and on the ethics – is welcome: github.com/Eight7Six/insta-facematch