Classic Machine Learning Through the Eyes of an SRE — Part 8
The most dangerous output in my whole Week-1 study set wasn't a bad prediction. It was a beautiful tree.
Hierarchical clustering produces a dendrogram, that elegant diagram where every account, ticket, or incident nests inside ever-larger families. It looks like discovered truth. Stakeholders lean in. Someone screenshots it for the QBR deck.
Nothing else in the set looks as convincing while being as capable of being completely wrong. A bad K-Means gives you blobs that feel arbitrary, and people push back. A dendrogram built with the wrong linkage on flat data still looks like a family tree of your business.
Nobody pushes back on a tree.
The bet and the build
Hierarchical clustering completes the answer-finding taxonomy I've been using through this series. That's my own shorthand, not standard terminology: K-Means SEARCHES, DBSCAN DEFINES, PCA SOLVES, and hierarchical clustering BUILDS.
Start with every point as its own cluster. Repeatedly merge the closest two clusters. Never undo. Greedy and irreversible, a little like growing a decision tree. Same skeleton, different family.
There is also a top-down version, called divisive clustering, which starts with everything together and splits it. In practice, when people say hierarchical clustering, they're usually talking about the bottom-up, agglomerative version.
Two things were genuinely new to me.
You choose the cut after seeing the structure. Fitting doesn't require you to decide K upfront. The dendrogram gives you the hierarchy, and you choose where to cut it to get the number of clusters you want. That makes the output unusually flexible. For a delivery organization it also feels natural, because account family → sub-segment → individual account is already how a lot of governance gets organized.
Linkage is a selectable worldview. "Closest clusters" needs a definition, and every definition makes a different assumption. Ward pushes toward compact, variance-minimizing clusters. Single linkage is comfortable with chains. Complete linkage favors tighter groups. You aren't configuring a minor implementation detail. You're choosing what kind of structure the tree is allowed to see.
Single linkage had another connection I immediately recognized. It can be understood through a minimum spanning tree: connect the points through the cheapest edges, then cut the longest ones. Once I saw that, chaining stopped being a quirk to memorize. A thin bridge of intermediate points can connect two otherwise separate groups, because the algorithm only needs those cheap nearest-neighbor links. That's a graph algorithm I already knew, wearing a clustering costume. The same thing happened in Part 6, where DBSCAN turned out to have the skeleton of connected components.
The subtlety that took me three rounds
What does hierarchical clustering optimize? My first answer was "nothing, like DBSCAN." Wrong. The optimization story is hybrid.
Agglomerative clustering doesn't optimize one global objective across every possible tree. It makes greedy merges according to the linkage you selected. Ward is the interesting case: each merge is chosen to minimize the increase in within-cluster sum of squares, the same quantity underlying the K-Means objective. Same objective family, different strategy. K-Means searches for a solution, Ward builds one greedily, and neither guarantees the globally optimal result.
One important restriction: Ward is tied to Euclidean distance. If you're thinking about cosine similarity or another non-Euclidean distance, Ward isn't the appropriate choice, and current implementations explicitly restrict it to Euclidean/L2.
That is when "what's the loss?" stopped being a checkbox question for me. The answer can be no single global objective, a linkage-specific criterion, or a greedy local criterion, and each one creates a different failure mode.
The part of the picture that isn't information
Here is the thing that changed how I read every dendrogram since.
The left-to-right position of the leaves is not a similarity measure. At every merge, the two child subtrees can be placed on either side without changing the underlying clustering, so the same hierarchy has many valid visual arrangements. What carries information is the vertical axis: how far apart the groups were when they merged.
But that's not how people read it. They scan a dendrogram horizontally, like a spectrum, and conclude that neighboring leaves are similar. Two accounts sitting side by side at the bottom may not be similar at all. They may only merge near the top of the tree, which is the algorithm telling you they are far apart. Libraries even provide leaf-ordering options to make the visual structure more intuitive without changing the clustering.
So the most persuasive axis on the most persuasive chart carries no similarity information at all.
That's what "fails beautifully" actually means.
The two checks worth running
Two useful checks tell you whether a beautiful tree deserves more scrutiny.
Cophenetic correlation compares the original pairwise distances with the distances represented by the dendrogram. High correlation means the tree preserves the pairwise geometry reasonably well. Low means you're looking at a substantial distortion.
Cluster stability means resampling the data, refitting, and checking whether the clusters survive. If the membership changes substantially across resamples, the structure isn't robust enough to treat as established fact.
A dendrogram that fails both is still gorgeous. That's the trap.
Two more practical notes. Distances are scale-sensitive, so standardize before fitting, the same rule that applied to K-Means, SVM and PCA earlier in the series. And when candidate merges tie, implementation and input ordering can affect which merge is selected, so two implementations can produce different trees from the same data. Part 6 had a version of this same lesson with DBSCAN border points, and I keep relearning it: "deterministic" usually has an asterisk.
The wall this hits
An n-by-n distance matrix is O(n²) memory. Depending on the linkage and implementation, agglomerative clustering can range from O(n²) to O(n³) time. SciPy documents O(n²) implementations for several common linkages and O(n³) for some others, with O(n²) memory across these implementations.
Fine for two hundred accounts. Not so fine for two hundred thousand tickets.
A mistake I made while studying and want to keep visible: I originally listed the scale wall as a trust check. It isn't. It tells you whether hierarchical clustering can run at all, not whether the answer is right. Different question entirely.
What I'd tell my ops team
Use hierarchical clustering when the domain is genuinely nested and reasonably small: account taxonomies, competency trees, incident-catalog dedup. Pick linkage deliberately, because it is your shape hypothesis, not a default. Run both checks before any stakeholder sees the tree. Never let anyone read left-to-right adjacency as similarity, and say that explicitly when you present it.
And put the dendrogram last in the deck, after the caveats. Because the moment it appears, the room stops evaluating and starts believing.
The convincing failure is the expensive one.
Production takeaway
Run both checks before a stakeholder sees the dendrogram: cophenetic correlation and cluster stability across resamples. Choose linkage deliberately. Standardize first. And know the scale wall, because O(n²) memory can become the constraint long before the business question gets interesting.
Common interview mistake
Two. First, saying hierarchical clustering optimizes nothing. The optimization story is hybrid: agglomerative clustering is greedy and linkage-driven, while Ward chooses each merge by minimizing the increase in within-cluster variance, related directly to the K-Means objective.
Second, treating the horizontal position of leaves as a similarity measure. It isn't. The tree's merge heights carry the distance information. Leaf ordering is primarily a visualization choice.
Where I'd use this in a real production system
Small, genuinely nested domains: account taxonomies, competency trees, incident-catalog dedup. Hundreds of items, not hundreds of thousands.
Classic Machine Learning Through the Eyes of an SRE. What each algorithm bets about your world, read through a production lens. Previous: PCA. Final part: Isolation Forest, the anomaly detector that builds isolation into the algorithm rather than optimizing a conventional loss.
