Metrics Governance Playbook and Certification Process

• Why single definitions end debates and save weeks
• Roles, RACI metrics, and the approval workflow that scales
• Certification criteria, metric templates, and SLA guardrails
• Onboarding, audits, and the lifecycle that keeps metrics true
• Practical application: templates, checklists, and CI/CD patterns

Conflicting KPI numbers stop decisions; they are not a people problem, they are a systems problem. A disciplined metrics governance program—backed by a semantic layer and a repeatable metric certification process—turns argument into action and meetings into decisions.

The symptoms are familiar: finance and product report different revenue numbers, dashboards show different conversion rates, and every review meeting starts with a reconciliation exercise. Behind those symptoms lie three causes: duplicated calculation logic across tools, missing ownership, and no objective, machine-checkable certification process. The result is wasted analyst hours, delayed decisions, and eroded trust in your data.

Why single definitions end debates and save weeks

• Principle: Define once, use everywhere. A semantic layer that houses canonical metric definitions reduces duplication, ensures consistency, and lets you treat metrics like code—versioned, reviewed, and testable. This is the core idea behind modern semantic layers such as dbt’s Semantic Layer.
• Metrics-as-code: Store metric definitions in YAML or similar artifacts, run them through PRs, and enforce tests in CI. That approach makes every change auditable and reversible, and lets you trace a dashboard number back to a single source-of-truth. MetricFlow is the engine DBT uses to compile YAML metric specs into SQL and enforce consistency.
• Tool-agnostic consumption: A headless semantic layer avoids BI lock-in by letting Looker, Tableau, Power BI, notebooks, or AI agents consume the same metric definition. BI-native modeling (e.g., LookML) has benefits when you’re Looker-first, but it stops scaling across heterogeneous stacks; a central semantic layer removes that single-tool bottleneck.
• Contrarian insight: Centralization will fail without delegated ownership. Centralized metric logic must pair with domain owners who hold accountability, not gatekeepers who become bottlenecks. Certification gates should protect stability, not slow every change to a crawl.
• Short example: Treat monthly_recurring_revenue as a code object. The business owner verifies the business rule, the analytics engineer implements the SQL and tests, CI runs end-to-end checks, and the catalog publishes a certified artifact that dashboards must reference. That flow removes ad-hoc spreadsheet logic and one-off SQLs.

Roles, RACI metrics, and the approval workflow that scales

Clear role definitions reduce churn. Use a RACI model that maps responsibilities for every stage of a metric’s lifecycle: definition, implementation, testing, certification, publishing, dashboarding, and monitoring. RACI remains a practical baseline for accountability and communication.

Notes on the table above:

• R = Responsible (does the work). A = Accountable (approver). C = Consulted. I = Informed. Use a single Accountable where possible to avoid split authority.
• Implementation pattern: changes live in a git repo (metrics-as-code), submit a PR, CI runs dbt sl validate and dbt test (or equivalent metric validations), AE and DE resolve technical issues, then Domain Owner approves the business semantics, then GC issues certification. MetricFlow and dbt provide commands and validations to embed into the CI pipeline.
• Practical automation: use the catalog as the approval UI (submit a certification request from the catalog); map catalog approvals back to the PR so that the entire audit trail lives in git and the catalog. Catalogs and governance platforms typically expose certificateStatus fields and can be updated by workflow automation.

Workflow (one-line flow you can implement today)

1. Open PR with metric change + embed metric_spec.yml.
2. CI: dbt sl validate (semantic validation), run dbt test and data quality expectations.
3. AE triages technical failures; push fixes to same PR.
4. Domain Owner performs business review in the catalog UI and marks "Business Approved."
5. Governance Council performs policy/compliance checks; if satisfied, they issue a Certified badge in the catalog.
6. BI tooling is configured to prefer or require certified metrics when building dashboards.

Certification criteria, metric templates, and SLA guardrails

Certification must be objective and largely automatable. A compact list of must-pass gates covers correctness, reproducibility, performance, and governance.

Minimum certification criteria (objective gates)

• Business definition present: plain-language description, owner, intended use, valid time window, and edge cases (e.g., refunds). Evidence: filled description + owner fields in the catalog.
• Canonical SQL / Expression: executable SQL or expression in the semantic layer with references to canonical models (no ad-hoc joins in dashboards). Evidence: PR + compiled SQL.
• Automated tests pass: unit and integration tests (e.g., null/uniqueness/freshness) executed in CI; structured data quality expectations for distribution/drift. Tools like Great Expectations provide expectations and metric storage that fit into validation pipelines.
• Lineage & provenance: clear upstream lineage from source tables to metric; version history available for audit. Evidence: lineage graph in the catalog.
• Performance and cardinality guardrails: query completes within agreed latency or has a pre-aggregated alternative. Evidence: performance test or cached materialization.
• Regulatory/compliance review: PII handling, retention, and masking validated if metric touches sensitive data. Evidence: compliance sign-off recorded in catalog.

Metric certification template (YAML — dbt/MetricFlow style)

# metrics/finance_metrics.yml
semantic_models:
  - name: orders
    model: ref('fct_orders')
    entities:
      - customer_id
    dimensions:
      - name: country
        sql: ${TABLE}.country

metrics:
  - name: monthly_recurring_revenue
    display_name: "Monthly Recurring Revenue (MRR)"
    description: |
      Total recurring revenue recognized in the month. Excludes one-time charges and refunds.
    metric_expression:
      language: SQL
      code: >
        SELECT
          DATE_TRUNC('month', order_date) AS month,
          SUM(CASE WHEN subscription = TRUE THEN amount ELSE 0 END) AS mrr
        FROM {{ ref('fct_orders') }}
        WHERE order_status = 'completed'
    unitOfMeasurement: DOLLARS
    metricType: SUM
    granularity: MONTH
    dimensions: [country, product_line]
    owners:
      - team: Finance
        person: finance_lead@example.com
    tests:
      - dbt: not_null: subscription_id
      - ge_expectation: expect_column_values_to_be_between: {column: mrr, min_value: 0}
    certification:
      status: pending
      requested_by: alice@example.com
      requested_at: 2025-12-01T10:00:00Z

This template reflects fields recommended in catalog standards and enables automated validation and publishing. Use metric_expression and owners as structured fields so tooling can parse and surface them.

Certification SLA guardrails (recommended)
| Step | Target SLA |
|---|---:|
| Triage (initial tech review) | 2 business days |
| Technical validation (AE + CI) | 5 business days |
| Business review (Domain Owner) | 5–7 business days |
| Governance review & certification | 3 business days |
| Total typical time (end-to-end) | 10–17 business days |

Set these SLAs as default service targets in the catalog ticketing flow; escalate exceptions for Tier 1 metrics with an expedited path.

Onboarding, audits, and the lifecycle that keeps metrics true

Onboarding blueprint (first 90 days)

1. Inventory: export all dashboards, extract metric names, and map to candidate canonical metrics. Use metadata scraping from BI tools and the catalog.
2. Prioritize: rank metrics by business impact (finance metrics, retention, revenue, LTV), usage frequency, and risk. Focus the first wave on top 10–25 high-impact metrics.
3. Pilot & migrate: implement canonical definitions in the semantic layer for the first wave, update 1–2 flagship dashboards to consume certified metrics, and measure delta in reconciliation time.
4. Rollout: migrate remaining dashboards in priority waves and update governance docs and training.

Audit cadence and triggers

• Tier 1 metrics (financial, legal): monthly automated checks + quarterly governance review.
• Tier 2 metrics (product, growth): weekly or monthly automated checks + quarterly review.
• Tier 3 (operational/low-risk): monthly automated checks + annual review.
• Trigger immediate re-certification when: data-quality tests fail, upstream schema changes, or business logic changes. Store run results and test-history; use coverage dashboards to track what percent of metrics have recent validations. Great Expectations and its coverage health metrics give a pattern for measuring test coverage and freshness.

Maintenance lifecycle (practical rules)

• Treat metrics like software: require PRs for changes, use branches for experimental metrics, and require rollback plans for any change to a certified metric.
• Auto-downgrade policy: a certified metric that fails critical tests should be automatically marked as temporarily uncertified in the catalog and notify owners; governance then rescues or remediates. Use your catalog’s certificateStatus field and automation hooks to implement this pattern.
• Retirement: metrics not referenced by any dashboard or report for 12 months move to deprecated state and are scheduled for deletion after owner confirmation.

Practical application: templates, checklists, and CI/CD patterns

Checklist: Certification request (must be attached to every PR)

• [ ] Business description and owner assigned.
• [ ] Canonical SQL/expression present and references only canonical models.
• [ ] Unit tests (not_null, unique, relationship) in dbt or Great Expectations.
• [ ] Performance test or materialization plan for heavy aggregations.
• [ ] Lineage included (upstream tables and transformations).
• [ ] Compliance review (if sensitive data).
• [ ] Example dashboard queries that will use the metric (to validate granularity/dimensions).

PR review checklist for AEs & DPMs

• Confirm the SQL compiles and returns expected cardinalities.
• Validate test coverage and run CI artifacts (manifest, test results).
• Confirm domain-owner comment / signoff in the PR.
• Confirm governance check (data sensitivity, retention).

Sample GitHub Actions CI snippet (run on PRs)

name: dbt Semantic Layer CI
on:
  pull_request:
    branches: [ main ]
jobs:
  validate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Set up Python
        uses: actions/setup-python@v4
        with:
          python-version: '3.10'
      - name: Install dependencies
        run: pip install dbt-core dbt-postgres metricflow
      - name: Semantic layer validate
        run: dbt sl validate
      - name: Run dbt tests
        run: dbt test --profiles-dir ./ci
      - name: Upload artifacts
        uses: actions/upload-artifact@v4
        with:
          name: dbt-manifest
          path: ./target/manifest.json

This pattern follows common CI/CD practices for dbt projects and semantic-layer validation; Snowflake’s guidance on dbt CI/CD shows similar staging and deploy patterns you can adapt to other platforms.

PR template (short)

## Metric change summary
- Metric: `monthly_recurring_revenue`
- Reason for change: Clarify treatment of refunds
- Owner: finance_lead@example.com

## Tests included
- dbt tests: not_null(subscription_id), unique(subscription_id)
- GE expectations: freshness (max_age=24h)

## Business approval
- @finance_lead: [ ] Approved

## Governance
- Compliance review: [ ] Completed

Governance automation suggestions (implementation notes)

• Wire the catalog to your CI: when a PR merges and tests pass, auto-update the catalog entry via API to reflect new version and last_certified_by fields. Catalog APIs and open standards (e.g., OpenMetadata/OpenMetric schemas) make this integration straightforward.
• Surface certification badges in BI: configure Looker or other BI tools to show "Certified" badges in field descriptions and to prefer certified metrics in explores.

A short runbook for metric incidents

1. Alert fires (test failed or drift detected).
2. Auto-change catalog certification.status → uncertified and page owner(s).
3. Owner triages, opens PR with fix, marks PR with hotfix tag.
4. AE applies fix in staging, CI runs, business verifies sample numbers, GC re-certifies.
5. Re-publish and notify downstream dashboard owners.

Sources

dbt Semantic Layer - Documentation describing the dbt Semantic Layer, how metric definitions are centralized in dbt, and the consumption/integration model for downstream tools.

About MetricFlow (dbt) - Technical overview of MetricFlow, the YAML metric abstractions, and the CLI/validation commands used to compile and validate semantic metric definitions.

Great Expectations — MetricStore & Coverage Health - Documentation on expectations, metric storage, and coverage/health concepts for data quality testing and monitoring.

OpenMetadata Metric Schema - Metric entity schema and recommended fields (description, metricExpression, owners, lineage, versioning), used as a reference for catalog metadata and certification fields.

Atlassian — RACI Chart: What it is & How to Use - Practical guidance on RACI roles and examples for mapping responsibilities across activities.

Looker product overview & semantic modelling - Documentation and product guidance describing Looker’s modeling layer (LookML), governance features, and how BI platforms surface modeled metrics.

Snowflake — CI/CD integrations on dbt Projects - Example patterns for integrating dbt projects into CI/CD pipelines, including PR validation and production deployment flows.

GitHub Actions — Workflows and actions reference - Official reference for defining workflow YAML files, triggers, and best-practice CI patterns for pull-request validation and deployments.

Alation — What Is Metadata? Types, Frameworks & Best Practices - Discussion of metadata management, certification/badging in catalogs, and how catalogs support governance, discovery, and trust.