When the Same Reference Data Lives in Three Services: Why We Moved It into a Dedicated Service

How to reduce data collisions when three services use different databases but rely on the same business semantics.

In distributed systems, problems rarely begin with something dramatic. On paper, everything usually looks rational: separate services, separate databases, clear responsibility boundaries. But at some point, the system starts arguing with itself.

I had a case where we had to split balance-related data across three different databases.

The first database was used by the account display service. The second one powered the service that showed customer-level balances. The third stored current account balances. Each database had its own service because each domain had different read patterns, different performance requirements, and a different data representation model.

At first, that separation looked reasonable. The real problem was elsewhere: all three services needed the same reference data.

That was where the real complexity began.

Where the Logic Started Breaking

When the same reference data exists in multiple services, the system almost inevitably starts producing different versions of the truth.

In one place, a status has already been updated. In another, an old category is still there. In a third, a local mapping was once introduced as a temporary workaround and somehow survived multiple releases. Formally, the data still looks similar. In practice, the business meaning starts to diverge.

For users, this looks like a simple inconsistency between screens:

• the same account appears in one category in one view;
• gets aggregated using another attribute in a different view;
• and is displayed with yet another interpretation somewhere else.

For the engineering team, this is no longer a cosmetic issue. It becomes incidents, manual reconciliation, long investigations, and eventually a loss of trust in the data.

The Initial Setup

At the architecture level, the system looked roughly like this:

• the account service worked with its own database;
• the customer balance service worked with its own database;
• the current account balance service worked with its own database;
• each service used the same reference data: account types, statuses, product attributes, classifiers, and other reference entities.

On paper, this is a standard enterprise architecture pattern. But the moment you have multiple owners of the same business semantics, you start running into questions that are very hard to answer consistently.

Who is the source of truth for a reference entity? When should an update be considered applied? What happens if one system is already using the new version while another is still on the old one? Who owns the investigation when the same field is interpreted differently across systems?

If I answer honestly, at that stage we did not have one good answer to those questions.

Why Reference Data Is Not “Secondary Data”

This is a common design trap. Reference data is often treated as something secondary: not money, not transactions, not the main operational flow. So teams assume they can simply keep a local copy next to every service and sync it somehow later.

But in practice, reference data defines how the system interprets the primary data.

A balance by itself is just a number. Its meaning appears only in context: product type, account status, segment membership, aggregation attributes, and other business qualifiers. If that context differs across services, then even the same balance value stops meaning the same thing to the business.

At that point, this is no longer a technical inconsistency. It is a semantic inconsistency.

What Problems Started Showing Up

The issues did not appear as one major outage. They accumulated as a growing gray zone.

First, reference data versions drifted asynchronously. One service had already applied a change, another had not, and a third updated through a separate process. As a result, the same entity started looking different across user journeys.

Second, services developed different refresh rules. One updated on a schedule, another reacted to an event, and a third relied on a separate integration flow. Even if the source reference data is nominally the same, different delivery mechanisms quickly create drift.

Third, diagnostics became much harder. Any incident of the form “why is it one thing here and another there?” required investigating three databases, three services, three update histories, and often multiple teams. The worst part was that these incidents rarely looked like major failures. They lived as chronic operational pain.

The Options We Considered

Once it became clear that local duplication of reference data was getting too expensive, we considered several options.

Option 1. Keep Local Copies and Improve Synchronization

The most obvious path was to keep the architecture as is and simply make update delivery more disciplined.

The benefit is obvious: services keep local data, fast reads, and their familiar integration model.

The downside is that the ownership problem does not go away. Even with better synchronization, the system still has multiple places where the same reference data can be interpreted and changed differently.

Option 2. Create a Shared Reference Database

This is better than having copies in three different places. At least there is formally a single storage layer.

But this option often creates another problem: services start reading the same schema directly, rules get duplicated in consuming code, and the contract evolves without a clear owner. In the end, you may have one database, but you still do not have one place that actually owns reference data behavior.

Option 3. Build a Dedicated Reference Data Service

This was the option we ultimately chose.

The idea was simple: reference data stops being “something shared by everyone” and becomes its own domain with an explicit owner. One service is responsible for storage, versioning, validation, and change publication. The other services stop deciding for themselves which version of a reference entity is the correct one.

Why We Chose a Dedicated Reference Data Service

The key insight was this: our real problem was not data delivery by itself. It was multiple owners of the same business semantics.

As long as reference data lives in several services, the system is almost guaranteed to produce local interpretations:

• one service introduces an extra status;
• another adds a local mapping;
• a third implements a special filtering rule;
• a fourth leaves a temporary workaround in place until it quietly becomes permanent.

A dedicated reference data service does not magically make the system simple. But it removes the main source of collisions: multiple decision centers for the same reference entities.

Put differently, we stopped synchronizing three almost identical truths and started operating with one.

What Changed in the Architecture

After we introduced the service, the architecture became conceptually simpler:

• the reference data service became the owner of the canonical model;
• the account service, customer balance service, and current balance service stopped owning reference data independently;
• validation, versioning, and update rules were centralized;
• all collisions were investigated against one source of truth rather than three local copies.

The important nuance is that we centralized ownership, not necessarily every online read.

If every consumer depends on synchronous runtime calls to the reference data service, then instead of data inconsistency you may end up with cascading degradation when that service is unavailable. In systems like this, the better pattern is usually to centralize the write model and the rules, while shaping the read model according to each service’s load profile.

What a Real Reference Data Service Must Do

If you build this service seriously, not as a thin CRUD wrapper over a table, it has several non-negotiable responsibilities.

First, it must own the model. Not just store data, but define fields, statuses, relationships, constraints, and lifecycle rules.

Second, it must support versioning. At any moment, a consumer should be able to answer which version of a reference dataset it is currently using.

Third, it must publish changes predictably. That can be done through APIs, events, snapshots, or a hybrid model, but the delivery mechanism needs to be controlled and consistent.

Fourth, it must enforce data quality: validation, auditability, deduplication, mandatory field checks, and a transparent change process.

Fifth, it must be observable. If balances are interpreted through this reference data, then freshness, version, update errors, and consumer lag all become production metrics, not implementation details.

Trade-offs

We chose the third option because in our case the main cost was not read latency. It was the cost of inconsistent data semantics across systems.

Where Teams Usually Get It Wrong

The most common mistake is to call something a reference data service when it is really just a database with a REST wrapper.

If that service does not own the model, has no versioning, no controlled change lifecycle, and no transparent data distribution model, then it does not solve the original problem. It merely moves it somewhere else.

The second mistake is forcing all consumers to read reference data synchronously and only online. That quickly turns an infrastructure component into a single point of cascading degradation.

In practice, a hybrid model is usually the right one:

• a single service owns the reference data;
• changes are published centrally;
• consumers can keep local read-optimized projections where that makes sense for load and latency;
• the right to change and interpret reference data remains centralized.

Decision Log

Context. Three services worked with three separate databases but used the same reference data to interpret balances.

Problem. We were seeing collisions, version drift, and difficult incidents where different systems showed different values or categories for the same entity.

Options. Keep local copies and improve synchronization; create a shared database; build a dedicated reference data service.

Decision. Build a dedicated reference data service as the single owner of reference entities.

Why. The core issue was not only data delivery. It was multiple ownership of the same business semantics.

Consequences. Fewer collisions, simpler investigations, and a clearer evolution path for reference data. The price was a new critical service that had to be designed as part of the core architecture rather than as a supporting utility.

What I Would Explicitly Build Into This System

If I were designing a similar setup for production from the start, I would explicitly include the following:

• a single writer for reference data;
• explicit versions and publication timestamps;
• freshness controls on the consumer side;
• auditability and clear ownership;
• backward compatibility rules for contract changes;
• observability not only for service availability, but for consumer data freshness as well.

Final Takeaway

When multiple services use the same reference data, the problem very quickly stops being about where to store tables. It becomes a question of who owns the meaning of the data.

In our case, we had three databases, three services, and one shared problem: the same reference data lived in several places and started drifting over time. Moving that logic into a dedicated reference data service did not remove all system complexity, but it removed the most dangerous kind of complexity: competing versions of the truth.

The short version is this: in a distributed system, you can live with different databases, different services, and different read models. But if reference data defines how the business understands its core data, it needs a single owner.