One definition per metric
"Active users" was 41,000 in the board deck, 38,500 in the growth dashboard, and 44,200 in the email report. None were wrong. That's the problem.
The most expensive data problems aren't broken pipelines — they're metrics that quietly mean different things in different places. Each team that needed "active users" wrote their own query, made a reasonable choice about the window and the filters, and shipped. Now there are seven definitions and no way to know which one a given number used.
The fix is organizational, expressed in code
One metric, one definition, one place it's computed. Everything downstream references that, rather than recomputing from raw events. In practice this means a modeled metrics layer: the definition of "active user" lives once, with its window, its exclusions, and a comment explaining the choices, and every dashboard pulls from it.
- Define once. The logic for a metric exists in exactly one model.
- Document the choices. Why 28-day? Why exclude internal accounts? Write it next to the code.
- Forbid re-derivation. Dashboards consume the metric; they don't recompute it from events.
A metric with two definitions is two metrics wearing the same name. Naming them apart is half the cure.
Migrating the existing mess
You won't fix seven definitions in a sprint. Pick the metric that appears in the most senior decisions, model it correctly, point the most-viewed dashboard at it, and announce the number it now produces. The discrepancy becomes visible and the pressure to consolidate follows. Repeat for the next metric. Trust is rebuilt one number at a time.