Haydn Strauss

AI agents are great at building data pipelines that look like they work until you dig into the results.

Write-audit-publish (WAP) helps fix that. Stage the data, audit it against a declared contract, and only publish once every clause passes. Netflix popularized this pattern in 2017.

A pipeline that finishes successfully is not the same as one whose output is correct.

We’ve built a number of internal skills to make our own data pipelines safer, and this one felt useful enough to release as a free WAP skill for coding agents.

The first test was on Netflix’s Top 10 dataset. The initial run stopped at the gate. Our contract said every film should have “N/A” as the season title, but the agent found nine rows that didn’t match. The contract was wrong, not the data. We fixed it, started a fresh run, and the second attempt published cleanly, with the total reconciling to exactly 185,656,120,000 hours viewed.

We ran it again on an NFL play-by-play pipeline (converting play description strings into structured stat tables). It caught a parser bug that left 1,723 completed passes without matching receptions, exactly the kind of thing a "successful" run hides.

Below, we dig a bit more into how the skill works. Give it a read, or point your coding agent at this URL and try it yourself.

Anthropic recently published how it runs self-service analytics on Claude. One result caught my eye: context + skills took its analytics agent from 21% accuracy to consistently above 95%.

Highlighting that generating SQL is the easy part, the hard part is everything underneath it: canonical datasets, a semantic layer, lineage, maintained skills, and provenance on every answer.

That jump came from the foundation, not a bigger model. With the context right, the agent on top matters much less.

Why Agents Alone Fail

In addition to cost, three context problems keep coming up.

• Entity ambiguity. "Active users" or "revenue" has several definitions in the warehouse. The agent picks one and writes correct SQL against the wrong data.

• Staleness. The definition was right when written. Then the pipeline changed and the skill was never updated.

• Retrieval failure. The right definition exists somewhere, but the agent can't find it, or grabs the wrong version.

Two of these, staleness and retrieval, can't be fixed easily by prompting alone. They need the context to be a versioned, owned asset wired to the pipeline it describes.

Anthropic tried the shortcut of handing the agent the raw query corpus, and accuracy barely moved. As they put it: "The information was there, the agent saw it, and it still didn't use it."

We build AI systems for a living. In production today, that means LangGraph wired into Belvedere: governed pipelines, human approvals, audit trails, provenance, etc.

But for this project, I wanted to kick the tires on something new: Flue, the agent framework from the Astro team. I’ve long been a fan of Astro for web development, so when they released Flue, I wanted to take it for a spin.

Initially I went down the path of adding Flue to background tasks (bug ticket sync, feedback triage, opportunity discovery, etc), but then decided to build something a little more fun, an 'agentic analyst org' powered by Flue.

It's completely free to use if you want to head to https://www.belvederelabs.ai/ and try it out with your data.

Flue Analyst Org | Belvedere Labs: drop a CSV and get an analyst's answer in a couple of minutes.

The data: 3,759 international matches assembled by Belvedere

We did not hand-roll the dataset. We built it in Belvedere, the same governed pipeline system we use in production.

The pipeline ("World Cup International Match Dataset") is eight nodes and seven edges in the canvas that was assembled by connecting our source API and using the following prompt:

"Pull senior men's international football fixtures and per-match statistics from the API-Football REST API, then computes leak-free pre-match features with all available statistics"