Building an investing knowledge graph, part 3: same company, seven names

I noticed a problem because a traversal that had been working stopped returning results. The ASML supply chain path I'd tested a week earlier came back empty. I spent a while thinking I'd broken the query. The query was fine. What had happened was that a new batch of articles from a different source used "the Dutch lithography equipment maker" and "ASML Holding NV" without ever using the string "ASML." Both landed in the graph as separate nodes. The traversal starting from the ASML node I'd been accumulating edges on couldn't reach them.

Seven names for one company

When I checked the Samsung Electronics entry in the registry, it had resolved seven distinct aliases to a single entity:

"samsung"
"samsung electronics"
"samsung electronics co."
"samsung foundry"
"samsung's"
"the korean company"
"the largest memory chipmaker in the world"

The foundry alias is a judgment call I went back on twice -- Samsung Foundry is technically a division with its own reporting, but in this corpus it was mostly used as shorthand for the parent in manufacturing capacity articles, so I kept it merged. The descriptive ones at the bottom are wire service shorthand. An article that opened with "Samsung Electronics" in the headline would use "the Korean company" three paragraphs later. Standard string matching returns near-zero similarity between "the largest memory chipmaker in the world" and "samsung electronics." They share no tokens.

Before I had a proper setup for this, I was using token overlap to match entity mentions. The threshold was loose enough to catch most of the alias variations above. It was also loose enough to merge Samsung SDI into Samsung Electronics.

Samsung SDI makes batteries. It is a separate publicly listed company. A lithium cell supply disruption hitting Samsung SDI's production line has nothing to do with Samsung Electronics' memory fab utilization. If those two ended up as the same node, events from one would show up in traversals that started from the other. I caught this specific case because I happened to be running a check on Samsung when a battery shortage article came through. Whether there are other pairs like this that I haven't caught, I genuinely don't know.

The descriptive ASML alias fails because it shares no tokens with the canonical name. Samsung SDI fails to separate because it shares the most salient token with an entity it should be distinct from. There's no threshold value that handles both. Loosening the threshold enough to catch the first kind of failure causes more of the second. These are actually different problems.

What Splink does with this

Splink treats entity resolution as a classification problem. For any candidate pair of mentions, the model computes a match probability using multiple features simultaneously and learns weights from a training set of confirmed pairs and non-matches.

For corporate entities in news text, the features I ended up using: canonical name similarity is the obvious starting point -- it handles "Samsung Electronics Co." but not "the largest memory chipmaker in the world." For descriptive references, alias coverage matters more: once a mention form has been confirmed as belonging to a given entity, future occurrences match by lookup. The ASML case gets better over time this way, even though the string overlap never improves.

For the Samsung SDI case, what made the difference was article-level context. An article that mentions DRAM yields and fab capacity almost certainly isn't about the battery subsidiary. Whether that context gets encoded as co-occurring industry terms or co-mentioned company names is a feature engineering choice; either way, it's information that string matching on the entity name alone can't see. I also added a base-rate feature after a few early merges looked wrong -- a rare entity with one mention shouldn't get absorbed into a high-frequency entity just because they share a token.

Splink learned weights across all of this from a training set of a few hundred manually verified pairs from the early corpus. Not large. But enough to get the Samsung SDI case right once I included the context features, which I hadn't initially.

The registry now has 47,853 resolved entities. That number has been climbing -- each new article either matches an existing entity or creates a new node that might get merged later as evidence accumulates. A few merges from early on are probably still wrong. Some entries I initially kept separate have since merged when more articles came in.

Still a threshold

Splink produces a probability, not a final answer. You still decide where to cut.

For this graph, I set that conservatively. A pair below a certain value stays split. Some valid aliases end up never linking to their entity -- nodes with few edges that mostly sit disconnected. I ran a check on a sample and most looked like genuine unknowns where there wasn't enough article evidence to confirm the connection. A few were probably real aliases I missed. That's the tradeoff: some fragmentation, in exchange for higher confidence that a merged entity represents one thing.

How that threshold gets set in practice, and what happens to the registry as new articles change the evidence picture, is part 4.

Built on Splink for probabilistic record linkage. Part 1 is here. Part 2 is here.