Most data teams that adopt a validation framework end up in the same place after a few months: hundreds of checks running, a channel full of alerts nobody reads, and no faster answer to the only question that matters when something breaks, which is who is affected. Adding more assertions does not fix that. The gap is architectural, not a matter of coverage.
Three tools cover the problem when each one does what it is good at. Great Expectations carries the deep assertions, dbt tests carry the cheap ones next to the transformation logic, and DataHub carries the results and the lineage that turns a failure into an impact assessment. Used separately they overlap and compete. Used together they form a loop: detect, locate, communicate.
This article covers where each tool belongs, how to route results so alerts stay actionable, and what to measure once the three are wired. It assumes you already know why data quality matters and want the integration, not the case for it.
Division of labor: which check goes where
The most common mistake is putting every check in one place. dbt tests are trivial to write and run inside the transformation, so teams write everything there and end up with a dbt build that takes an hour. Or they push everything into Great Expectations and lose the tight coupling between a model and its own assumptions.
| Layer | Tool | What it should assert | Cost of running |
|---|---|---|---|
| Ingestion | Great Expectations | Schema conformity, type validity, null rates, value ranges on raw inputs | Moderate, isolated from the warehouse |
| Transformation | dbt tests | Uniqueness, referential integrity, accepted values, model-specific business rules | Cheap, runs with the model |
| Curated output | Great Expectations | Distribution shifts, cross-table reconciliation, row count expectations against a source of record | Higher, worth it on few tables |
| Metadata layer | DataHub | Nothing is asserted here; results and lineage are collected and exposed | Low, read-heavy |
The rule that keeps this stable: a check belongs where the failure originates, not where it is noticed. A null in a warehouse column caused by a broken upstream API belongs at ingestion. A referential break introduced by a join belongs in the dbt model that performs the join. When checks drift away from their cause, every failure becomes an investigation.
dbt's not_null, unique, relationships and accepted_values cover a surprising share of real incidents at almost no cost, and its documentation covers the generic and singular test patterns. Reach for Great Expectations when the assertion needs statistical context, a comparison across systems, or a vocabulary your dbt tests cannot express.
Wiring results into DataHub
This is the step most implementations skip, and it is what separates a validation suite from a quality practice. Both Great Expectations and dbt can emit their results as metadata rather than only as pass/fail in a CI log.
Great Expectations ships a DataHub action that publishes validation results as assertions attached to the dataset entity. dbt test results reach DataHub through its dbt ingestion source, which also carries model documentation and the dbt-declared lineage. Once both are flowing, each table in the catalog shows its assertions, their recent outcomes and its upstream and downstream dependencies in one place. That combination is what makes DataHub and dbt work as a governance layer rather than as a searchable table list.
Why lineage changes the response
Without lineage, a failed check tells you a table is wrong. With lineage, it tells you which dashboards, models and downstream consumers depend on that table, which converts a technical alert into a decision about who to notify and what to pause.
That is the practical difference between knowing about a failure and being able to act on it. A freshness violation on a staging table with no downstream consumers is a ticket for tomorrow. The same violation on a table feeding a finance dashboard is a call now. Only the lineage graph can tell those two apart, and no amount of additional assertions will.
Killing alert fatigue
Once hundreds of checks run, the failure mode shifts from missing problems to ignoring them. Two design decisions prevent that, and both are about routing rather than about the checks themselves.
First, separate severity from the assertion. A check that fails should carry a declared consequence: block the pipeline, warn and continue, or record only. In dbt this maps to test severity and thresholds; in Great Expectations it is a property of how the checkpoint result is handled. Most checks should warn, not block. A suite where everything blocks gets disabled the first time it stops a release on a rounding difference.
Second, route by lineage, not by table. Alerts sent to a general channel are read by nobody in particular. Alerts routed to the owner of the affected downstream asset are read by the person who can decide. DataHub carries ownership metadata precisely so this routing can be automated, and the same principle applies to how you design alerts and notifications anywhere in the stack.
The measure to watch is not how many checks pass. It is what share of alerts produced an action. When that share drops below roughly half, the suite is generating noise and the fix is to delete checks, lower severities or fix routing, not to add more assertions.
What to measure after the three are wired
Coverage is a weak metric because it rewards volume. Four numbers describe whether the practice is working:
Time to detection, meaning how long a bad record survives before a check catches it. Time to impact assessment, meaning how long it takes to know which consumers are affected, which is where lineage pays off. Share of actionable alerts, as above. And escaped defects, meaning incidents found by a business user rather than by the pipeline, which is the only metric that measures the gap in your coverage honestly.
Escaped defects are the most useful of the four. Every one of them is a specific assertion you did not write, and treating each as a mandatory addition to the suite is what makes coverage grow from evidence instead of from guesswork.
The integration described here is not about buying three tools. It is about deciding, once, that assertions live next to their cause, that results are metadata rather than log output, and that alerts route by dependency rather than by convenience. Teams that make those three decisions get a practice that compounds, because each incident leaves the system slightly better instrumented than before. Teams that skip them end up with the same hundreds of checks and the same unread channel, and conclude that data quality tooling does not work.
If your team is wiring data quality checks across a production pipeline and deciding how validation, transformation tests and metadata should fit together, our specialists can help you design an architecture that matches your stack. Talk to our team and move your data maturity forward. ⬇️
How do Great Expectations, dbt tests and DataHub work together?
Each covers a different layer. dbt tests assert cheap rules next to the transformation logic, Great Expectations carries deeper assertions at ingestion and on curated outputs, and DataHub collects the results as metadata alongside lineage. The combination turns a failed check into an impact assessment, because lineage shows which downstream consumers are affected.
Should data quality checks live in dbt tests or in Great Expectations?
A check belongs where the failure originates. Uniqueness, referential integrity and accepted values belong in the dbt model that could break them. Schema conformity and null rates on raw inputs belong at ingestion, in Great Expectations. Statistical checks and cross-system reconciliation also belong in Great Expectations, because dbt tests cannot express them.
How do you avoid alert fatigue with hundreds of data quality checks?
Separate severity from the assertion so most checks warn instead of blocking, and route alerts by lineage to the owner of the affected downstream asset rather than to a general channel. Track the share of alerts that produced an action: below roughly half, the fix is deleting checks or fixing routing, not adding more.
What does DataHub add that a validation framework does not?
Lineage and ownership. A validation framework tells you a table is wrong; DataHub tells you which dashboards and models depend on it and who owns them. That converts a technical alert into a decision about who to notify and what to pause, which is what makes the difference between detecting a failure and responding to it.
Which data quality metrics are worth tracking in production?
Time to detection, time to impact assessment, share of actionable alerts, and escaped defects, meaning incidents found by a business user rather than by a check. Escaped defects are the most useful, because each one identifies a specific assertion missing from the suite and lets coverage grow from evidence instead of guesswork.
