AI infrastructure Living documentation Semantic layer

Ask the model what the metric means

Definitions lived in people's heads and three teams quoted three different completion rates. Built a copilot over the semantic layer so anyone can ask what a metric means and get an answer grounded in the live model, with its lineage.

One

definition, the same in every team

Days

to onboard, instead of weeks of asking around

From code

docs regenerate from the model, so they cannot drift

Self-serve

analysts answer their own definition questions

The client

An online learning platform whose data team supported product, content and partnerships, where the same metric names meant different things depending on who you asked.

The problem

The knowledge of what "active learner" or "completion" actually meant lived with two senior people. Documentation existed but had quietly fallen out of step with the model. New hires lost weeks asking around, and "whose completion rate is right" kept reaching the leadership meeting. The model was correct; what it meant was not legible.

How it ran

Documentation that reads from the model instead of describing it from memory.

Step 1

Reads the model

It indexes the semantic layer: the metric definitions, the SQL, and where each is used.

Step 2

Answers in plain English

Ask what a metric means; it explains the definition the way a senior analyst would.

Step 3

Shows its source

Every answer points at the exact model object it came from, so it is checkable.

Step 4

Regenerates

When the model changes, the documentation re-derives. There is no second copy to keep in sync.

Step 5

Coaches the team

Analysts use it to answer their own questions, so the two stop being the bottleneck.

What I did

01.Indexed the semantic layer: metric definitions, the SQL behind them, and every place each metric is consumed.
02.Built a copilot that answers "what does this mean / how is it computed / which table" from that index, in plain language.
03.Grounded every answer in the live model with a link to the exact object, so it can be verified rather than trusted blindly.
04.Wired regeneration to the model so documentation re-derives on change and cannot fall out of step.
05.Rolled it out as the first stop for definition questions, so the analysts stopped being a lookup service.
06.Maintained it on retainer: as the model and the team evolve, the copilot keeps tracking them.

Under the hood

How it stays trustworthy

•Retrieval is structured. It keys on the metric object in the semantic layer, so "completion" and "course completion" cannot collide the way fuzzy text search would.
•It regenerates from the model. A change to the model triggers regeneration through a hook on the model repository, so there is no second copy to fall out of step.
•It grounds every answer or it declines. Each answer points at the model object it came from; a question it cannot ground, it refuses.
•It frees the experts rather than speaking for them. It takes the lookup questions so they spend their time on the work only they can do.

What it will not do

It will not answer a definition it cannot point at a model object for. An ungrounded guess is worse than a refusal, so it refuses.

InteractiveTry it

Ask about a metric. Flip to the old static doc to see the drift.

Sample definitions, illustrative only

Definition copilot

Result

What the team got.

One definition everywhere

Product, content and partnerships read the same answer, because it comes from the one model.

Onboarding in days

A new analyst asks the copilot instead of booking time with the two people who hold it all.

Docs that cannot drift

They regenerate from the model, so the explanation always matches what actually runs.

It stays current

It keeps tracking the model and the team as both change.

Built as a fixed-scope engagement, then maintained on retainer so it keeps tracking the model. The deliverable includes your team able to run and change it after the engagement ends.

Tools used

BigQuery LookML / semantic layer Claude Retrieval over the model Docs as code

Three teams, three completion rates, one metric?

It usually starts as a short Spotcheck to size the work, then a fixed-scope build, then it is maintained on retainer so it keeps tracking the model.

See how a Spotcheck works Book a call