Building a Steward-Facing Lineage Report

Part of: Lineage Visualization and Reporting

A data steward is not an auditor and not an engineer. They own a handful of datasets, they have perhaps ten minutes a week for provenance, and what they need from a lineage system is a short answer to “is anything wrong with my data, and what should I do about it?” This how-to builds the report that answers that, which turns out to be a very different artefact from the compliance export and from the engineer’s traversal view.

The design constraint that shapes everything: the report must be readable in two minutes and must end with an action or with nothing. A report that requires interpretation will be skimmed; a report that always shows something amber will be ignored within a month.

Prerequisites

  • Dataset ownership recorded and reasonably current, per Lineage Visualization and Reporting.
  • Freshness, quality and completeness figures per dataset.
  • A traversal that can answer ancestry and impact per dataset.
  • Somewhere to deliver it that stewards already read.

Scope It to What They Own

The single biggest determinant of whether a lineage report gets read is whether it is about the reader’s own data.

Scope determines whether the report is read An organisation-wide report is ignored; a permission-scoped one is still too broad; an ownership-scoped one is short enough to act on. SCOPE ROWS OUTCOME everything in the org "the lineage dashboard" 2 400 nobody opens it twice everything they can see permission-scoped 310 skimmed, not acted on what they own ownership-scoped 9 read, and occasionally acted on — the goal

Nine rows is a report. Three hundred is a database view with a covering note, and stewards treat it as one.

Where ownership is unrecorded, that is the first thing the report should surface — to whoever commissioned it rather than to a steward who by definition does not exist. An unowned dataset with a quality problem has no addressee, and generating the report is how that gap becomes visible.

Include the datasets they own indirectly through a team, and say which is which. A steward who is on the hook for a colleague’s dataset while that colleague is on leave needs to see it, and needs to know it is not normally theirs.

Three Questions, In This Order

The report answers the same three questions every week, in descending order of urgency, and adding a fourth is how it becomes unreadable.

Is anything broken now? Failed runs, quality gate breaches, missing outputs. These have an action and a deadline.

Is anything about to break? Upstream changes landing, sources approaching end of support, freshness trending the wrong way. These have an owner to talk to.

Is anything drifting? Slowly falling completeness, growing manual corrections, an ancestry that has gained a step nobody remembers adding. These have no deadline and are the ones that turn into the first category if nobody looks.

Resist adding a summary score. A single health number is compelling to whoever specifies the report and useless to whoever reads it, because it collapses exactly the distinctions above. If a score is politically necessary, put it at the bottom.

Give Every Row an Action and an Addressee

A finding without an action is a puzzle, and stewards under time pressure skip puzzles.

A finding versus an actionable row The same defect expressed as a bare observation and as a row carrying cause, action and addressee. WHAT MOST REPORTS SAY parcels_v3 — completeness 91.4% (threshold 95%) — FAIL Now what? WHAT A STEWARD CAN USE parcels_v3 — completeness 91.4% (threshold 95%) Cause: 47 features dropped in Tuesday's update; 40 were logged retirements, 7 were not. Action: review the 7 unexplained drops (linked) and confirm or restore. Contact: J. Okonkwo, cadastral ingest — by 14 August. The extra four lines are all derived from data the system already holds.

The caption is the argument for doing this at all. Cause, action and contact are not editorial additions — they are the change-set reason, the propagation rule and the ownership record, all of which the lineage system already stores. Assembling them into a sentence is presentation work, not investigation work.

Link the evidence rather than embedding it. A row saying seven features dropped should link to those seven, so that verifying takes a click instead of a query. This is what turns a report from something to be believed into something to be checked.

Name a person, not a team, where one can be resolved. A row addressed to “cadastral ingest” is addressed to nobody, and a fortnight later everyone will assume somebody else picked it up.

Say Nothing When Nothing Is Wrong

The hardest discipline in periodic reporting is the empty report, and it is what preserves the report’s credibility.

A weekly report that always contains something amber teaches its readers that amber means normal, and after that a genuine amber is invisible. Set the thresholds so that a clean week produces a report saying so in one line.

Send the empty report rather than suppressing it. Its absence is ambiguous — the system might be broken — and a one-line “nine datasets, all clear” costs the reader three seconds and confirms the machinery is running.

Track the ratio of clean to non-clean reports over time. A report that has not been clean in four months has thresholds that are wrong or a genuine systemic problem, and either way it is not doing its job.

Deliver It Where They Already Are

A report on a dashboard nobody visits has the same value as no report.

Push it. Email, a chat channel, whatever the team already reads — the report should arrive rather than waiting to be fetched. The dashboard is where somebody goes after the push tells them to.

Make the push self-contained for the clean case and a link for the rest. Nobody should have to click through to learn that nothing is wrong.

Keep the delivery cadence matched to the response cadence. A daily report to someone who looks at their data weekly generates six ignored messages and one skimmed one; the weekly version generates one that is read.

Show the Change, Not Just the State

A weekly report describing the current state makes the reader do the diffing, and they will not.

Organise by change since the last report New, resolved and still-open sections let the reader attend only to what moved. NEW SINCE LAST WEEK — 2 parcels_v3 completeness · roads_ext upstream CRS changed RESOLVED — 3 no action needed; listed so the effort is visible STILL OPEN — 1 (4 weeks) the age is the point: a finding nobody closes needs escalating, not repeating

The age on the still-open section is what stops a report becoming wallpaper. A finding repeated identically for four weeks is telling you that either it cannot be fixed or nobody owns it, and both need a different response from repeating it a fifth time.

List resolved items even though they need no action. It shows the report is tracking rather than merely observing, and it gives the reader the only positive feedback the artefact ever provides.

Escalate automatically on age. A finding open past a stated threshold should appear in the manager’s report as well, without anybody having to decide to raise it.

Keep the change comparison against the previous report rather than against an arbitrary window, so that the arithmetic matches what the reader remembers seeing.

Verification

Assert scoping by generating for a steward with known holdings and confirming no row concerns anything else. Ownership leaks are both a privacy issue and the fastest way to make the report unreadable.

Assert the clean path produces a clean report, not an empty file or a suppressed message. Generate against a fixture where everything passes and check what actually gets sent.

Assert every non-clean row carries an action and a resolvable contact. A row failing that check should fail the report generation rather than being emitted incomplete.

Assert links resolve, including the evidence links. A report whose evidence links 404 is worse than one without them, because it invites a check that then fails.

Gotchas & edge cases

  • Two stewards on one dataset means neither acts. Where co-ownership is real, name a primary in the report rather than addressing both.
  • Ownership records rot faster than data does. Reconcile against the staff directory before sending, and route rows for departed owners somewhere a human sees them.
  • A steward with forty datasets is not a steward. If the scoped report is long, the ownership model has a problem the report is now surfacing. Say so rather than paginating.
  • Thresholds are per dataset, not global. A completeness target appropriate for a parcel layer will be permanently red on a voluntary-contribution dataset. Store the threshold with the dataset.
  • The report is a disclosure surface. It carries counts, contacts and sometimes locations. Apply the same access rules as the underlying data rather than assuming a summary is safe.
  • The first edition sets the expectation. A report that launches with twenty findings teaches its readers that it is a backlog rather than an alarm. Clear the known backlog separately before the first send, so the first report a steward sees is one they can act on entirely.
  • A report with no history has no baseline. Findings are only judgeable against what is normal for that dataset, so run in reporting-only mode for a few cycles before anything is presented as a breach.
  • Nobody reads the second page. If the content will not fit, cut by severity and state the count omitted; do not continue overleaf.