Blast radius is the set of downstream tables, models, and dashboards that become unreliable when one upstream table breaks. Lineage is how you compute it, and computing it is the difference between an incident you can triage and forty alerts you have to read.
Most writing about data lineage sells it as documentation. A pretty graph, a governance artifact, something you show an auditor. That framing is why lineage projects stall: nobody maintains documentation that does not pay for itself on a Tuesday afternoon.
Lineage pays for itself during an incident. That is the only argument for it I find convincing, and it is the one nobody makes.
What actually happens when an upstream table breaks?
Here is the shape of the problem, and if you have been on call for a data platform you already know it.
A Fivetran sync fails at 19:04. Nobody notices, because the sync failing is not the thing that pages you. At 02:00 the transformation job runs against stale source data and succeeds, because stale data is still data. At 06:30 the freshness monitors fire. By the time you open Slack there are thirty or forty alerts: raw_orders is stale, stg_orders is stale, fct_revenue is stale, the executive dashboard is stale, the finance extract is stale, three ML feature tables are stale.
Every one of those alerts is true. Not one of them tells you that a single Fivetran connector is the reason.
So you start reading. You open the lineage graph if you have one, or you grep the dbt project if you don't. You work backwards from the most alarming alert to whatever sits upstream of it. Fifteen minutes later you know what you could have known immediately: one connector broke, and everything else is a symptom.
That fifteen minutes is the tax. It gets paid on every multi-table incident, by whoever is on call, usually before coffee.
What does the triage tax actually cost?
Put numbers on the fifteen minutes, because "lineage saves time" is the kind of
claim every vendor makes and nobody quantifies.
Assume a loaded engineering cost of $80 to $150 an hour. Assume a multi-table
incident takes 15 minutes of pure root-cause hunting before any fixing starts,
which matches what I see on warehouses in the low hundreds of tables. Assume
those happen twice a month, which is conservative for a team shipping schema
changes weekly.
| Without lineage | With lineage and health | |
|---|---|---|
| Alerts to read | 30 to 40 | 1 incident |
| Time to identify root cause | ~15 min | under a minute |
| Who finds out numbers are bad | whoever opens the dashboard | you tell them first |
| Cost per incident at $115/hr | ~$29 | ~$2 |
| Cost per year at 24 incidents | ~$690 | ~$48 |
Six hundred dollars a year is not the argument. The argument is the fourth row.
The money is rounding error against an engineer's salary, and any vendor waving
that number at you is padding a business case. What actually matters is that
without a blast radius you find out your numbers were wrong when a stakeholder
tells you, and with one you tell the stakeholder first. That difference does not
show up in a spreadsheet and it is the entire reason to bother.
Two caveats on the arithmetic. The 15-minute figure is my estimate from watching
teams triage, not a benchmark, and it scales with how many tables you have and
how well you know them. If you have forty tables and wrote all of them, lineage
buys you very little. The value climbs steeply somewhere past the point where one
person can hold the graph in their head.
Why is table-level lineage enough for triage?
There is a persistent belief that lineage is only useful at column granularity. Column-level lineage is genuinely better for impact analysis before a change, when you want to know whether renaming one field breaks anything.
For incident response, table-level lineage does the job. When raw_orders stops updating, every column in it is stale. You do not need to know which columns feed which downstream fields, because the answer is all of them. The question during an incident is not "which fields are affected," it is "what do I have to tell people, and which pipeline do I fix first."
That distinction matters because table-level lineage is dramatically cheaper to obtain. You probably already have it and have not turned it on.
How do you get lineage without a migration?
If you use dbt, your lineage already exists. It is sitting in the manifest file dbt generates every time it compiles.
dbt parse # fastest, writes target/manifest.json without compiling
# any of these also write it
dbt compile
dbt run
manifest.json contains every model, source, seed, and snapshot in your project, plus the parent and child relationships between them. That is a dependency graph. Upload it and you have lineage.
In AnomalyArmor that means going to the Lineage tab for a database asset and uploading the file. Nodes come back as tables, edges as dbt dependencies, and each node carries the database, schema, and table name so it lines up with what you are actually monitoring.
A few things worth knowing before you do it:
- You do not need to change your dbt project. No new macros, no package, no config block. The manifest is a build artifact you already produce.
-
Sources count. Anything declared in a
sources:block shows up as a source node, so the graph extends past the models you wrote to the raw tables they read from. -
Re-upload after structural changes. The manifest is a snapshot. Add models and the graph does not know until you upload again. Wiring this into CI after
dbt parseis the obvious move.
Wiring the upload into CI is the part that keeps the graph honest:
# .github/workflows/dbt.yml
- name: Parse dbt project
run: dbt parse --target prod
- name: Upload lineage
run: |
curl -sf -X POST \
"$ARMOR_API/api/v1/assets/$ASSET_ID/lineage/upload" \
-H "Authorization: Bearer $ARMOR_API_KEY" \
-F "file=@target/manifest.json"
Run it on merges to main rather than on every pull request. The graph should
reflect what is deployed, not what someone is proposing.
If you do not use dbt, you can build the graph manually, which is worth it for a critical path and not worth it for a warehouse.
What does cascading staleness do?
This is the part that turns a graph into a triage tool.
A lineage graph on its own is structure. It tells you fct_revenue depends on stg_orders. Useful, static, and something you could have read out of the dbt project yourself.
The useful version carries health. Each node shows its own state: healthy, warning, critical, or at risk from upstream. When a node goes stale or critical, every node downstream of it is marked at risk automatically.
That single behavior collapses the forty-alert problem. Instead of reading thirty individual freshness failures, you look at the graph and see one red node with an orange fan spreading out from it. The red node is the thing to fix. The orange nodes are the blast radius, which is what you tell people.
The distinction between "at risk" and "critical" is doing real work here. An at-risk node has not failed a check of its own. It is downstream of something that did, so its data is suspect. Treating those as separate states is what stops the blast radius from looking like forty independent fires.
| Node state | What it means | What you do |
|---|---|---|
| Healthy | Its own checks pass, nothing upstream is broken | Nothing |
| Warning | A check is degraded but not failing | Watch it |
| Critical | Its own check failed | This is probably the root cause |
| At risk from upstream | Its checks pass, but something upstream is critical | Communicate, do not debug |
That last row is the one that saves time. Engineers waste triage minutes debugging tables that are working correctly and reporting bad numbers because their input is bad.
How do alerts stop being a flood?
Lineage plus health gets you a readable graph. The next step is not sending forty messages in the first place.
When a problem cascades through lineage, the downstream alerts collapse into a single incident with a named root cause and a visible blast radius. Slack gets one message rather than forty. Later alerts thread underneath it instead of arriving as new notifications. When every grouped alert clears, the incident auto-resolves on its own.
This is on by default for accounts that have lineage, which is the other reason to upload the manifest. Without a graph there is no way to know that fct_revenue failing and raw_orders failing are the same event.
Full details are in the incident correlation docs.
Manifest lineage or query-log lineage?
There are two ways to build a lineage graph and they fail differently, so it is
worth knowing which one you are buying.
Manifest-based lineage reads your transformation tool's own dependency
declarations. In dbt that is manifest.json. It is exact for everything dbt
manages, because dbt already resolved those dependencies to build your models in
the right order. It is completely blind to anything dbt does not manage.
Query-log lineage parses the SQL your warehouse actually executed and infers
dependencies from it. It catches everything that touched the warehouse regardless
of what tool wrote it, including the analyst's scheduled query and the Python job
nobody documented. It also inherits every ambiguity in SQL parsing: dynamic SQL,
SELECT * through views, and temp table chains all degrade it.
| Manifest | Query log | |
|---|---|---|
| Accuracy inside dbt | Exact | Inferred |
| Sees non-dbt writes | No | Yes |
| Setup cost | Upload a file | Warehouse log access and permissions |
| Fails by | Omitting what dbt does not manage | Mis-parsing unusual SQL |
| Freshness | As of last upload | Continuous |
We do manifest-based lineage. I am telling you that plainly because the honest
consequence is the one above: if a meaningful share of your tables are written by
something other than dbt, our graph will understate your blast radius, and a
graph that understates blast radius is worse than knowing you do not have one.
The practical answer for most teams is that dbt manages the transformation layer
and the gaps are at the edges: ingestion above it and reverse ETL below it.
Ingestion usually appears anyway, because dbt sources declare the raw tables.
Reverse ETL usually does not, so if a broken table feeds a sync back into
Salesforce, that consequence will not be in the graph and you need to know it
from memory.
Pick based on where your pipelines actually live rather than on which sounds more
sophisticated.
How should you triage with lineage?
Here is the sequence I would actually follow, and it fits on an index card.
1. Find the deepest critical node. Not the loudest alert, the one furthest upstream that is critical rather than at risk. That is your root cause candidate. In a graph with health, this is a visual step rather than an investigation.
2. Confirm it is a cause and not a coincidence. Two unrelated things do break at once occasionally. Check whether the at-risk set actually descends from that node. If half your alerts sit outside its downstream cone, you have two incidents.
3. Read the blast radius before you fix anything. The people who need to know are the owners of the at-risk leaves: the dashboard consumers, the feature store, the finance extract. Telling them at 06:35 that numbers are suspect is worth more than fixing the pipeline ten minutes sooner and telling them at 07:15.
4. Fix the root, then watch the fan clear. If the graph is doing its job, resolving the critical node should walk the at-risk states back to healthy as each downstream check re-runs. If a node stays at risk after its upstream recovers, that node has its own problem and you have found a second incident.
5. Record the time to detection. The gap between the Fivetran failure at 19:04 and the first alert at 06:30 is over eleven hours of undetected staleness. That number, not the fix time, is what you should be trying to shrink. We wrote about measuring it properly in data downtime.
The step most teams skip is 3. It is also the only one that changes how the incident feels to everyone who is not you.
What does lineage not tell you?
I would rather be useful than complete, so here are the limits.
It does not tell you why. The graph says raw_orders is critical. It does not say the Fivetran connector's credentials expired. Lineage narrows the search to one node; something else has to explain that node. Our investigations produce a cited capsule for exactly that gap, where each claim links back to the alert or record behind it.
It only knows what the manifest knows. A pipeline that writes to a table outside dbt is invisible. Reverse ETL, a Python job, a stored procedure, an analyst's scheduled query: none of it is in the graph unless you add it. Lineage graphs are always partial, and treating a partial graph as complete is how you conclude the blast radius is smaller than it is.
It is table-level. If you need to know whether renaming one column breaks a specific downstream field, this will not answer it. It will tell you which tables to go look at.
A stale manifest lies confidently. This is the failure mode I would watch for. A graph built from a three-month-old manifest will happily show you a blast radius that omits every model added since. It looks correct because graphs always look correct. Upload from CI.
Where do schema changes fit?
Freshness is the easy case, because a stale table is obviously stale. Schema changes are the ones that cascade quietly.
A column changing type does not stop a pipeline. Downstream models keep building, dashboards keep rendering, and the numbers are wrong until somebody notices. Lineage plus schema monitoring is what turns that into an alert with a blast radius attached instead of a discovery three weeks later. We covered the detection side in monitoring schema changes, and the dbt-specific version of quiet failure in silent dbt test failures.
Frequently asked questions
What is blast radius in data engineering?
The set of downstream tables, models, and dashboards that become unreliable when one upstream table breaks. It is computed by walking the lineage graph forward from the failed node.
Do I need column-level lineage for incident response?
No. When a table stops updating, every column in it is stale, so table-level granularity answers the triage question. Column-level lineage is more useful for impact analysis before making a change.
How do I get lineage from dbt?
Run dbt parse to produce target/manifest.json, then upload that file on the Lineage tab of your asset. dbt compile and dbt run write it too; dbt parse is fastest because it skips compilation. The manifest already contains every model, source, seed, and the dependencies between them.
Do I have to change my dbt project?
No. The manifest is a build artifact dbt produces during a normal compile. There is no package to install and no config to add.
How often should I re-upload the manifest?
After any structural change, which in practice means wiring it into CI after dbt parse. A stale manifest produces a graph that looks correct while omitting recently added models.
What if I do not use dbt?
You can define lineage manually. It is worth doing for a critical path of a dozen tables and not worth doing for an entire warehouse.
What does "at risk" mean on a node?
Its own checks pass, but something upstream of it is critical, so its data is suspect. It is a signal to communicate rather than to debug, because the node itself is working correctly.
Does the blast radius include dashboards?
It includes whatever is in the graph. Tables and models come from the manifest. BI dashboards appear if they are represented as nodes, otherwise the graph ends at the table that feeds them.
How does lineage reduce alert volume?
Downstream alerts that share a root cause collapse into one incident with a named cause and a visible blast radius, so a cascading failure produces a single Slack message instead of one per affected table.
Does the incident close itself?
Yes. Once every alert grouped under the incident clears, the incident auto-resolves.
What if two unrelated things break at once?
Check whether the at-risk set actually descends from your candidate root cause. Alerts outside that node's downstream cone belong to a separate incident.
Can lineage tell me the cause of a failure?
No. It narrows the search to a node. Explaining that node needs something else: logs, a schema event history, or an investigation that cites the underlying records.
Is a partial lineage graph still useful?
Yes, as long as you know it is partial. The risk is concluding the blast radius is smaller than it really is because a non-dbt pipeline is missing from the graph.
What should I measure to know this is working?
Time to detection, not time to resolution. Shrinking the gap between when a source actually broke and when the first alert fired is where the real hours are.
What about tables written outside dbt?
They will not appear in a manifest-based graph. Ingestion usually shows up anyway
because dbt sources declare the raw tables, but reverse ETL and ad-hoc jobs will
not, so a broken table feeding a Salesforce sync is a consequence you have to
remember rather than read off the graph.
