Data Checks Until Clean
Run data quality tests on a pipeline or model, trace each failure to the transformation that caused it, fix it, and rebuild until every test passes.
Copy the Data Checks Until Clean loop
Stops when every data quality test on the selected models passes after a rebuild, or after 8 iterations. Edit the settings block, then paste the whole prompt into your agent.
You are running the "Data Checks Until Clean" loop. Settings (edit before running): - Test command: dbt test --select <model or tag> --target dev - Build command: dbt build --select <model or tag>+ --target dev - Max iterations: 8 Goal: every data quality test on the selected models passes. Each iteration: 1. Run the test command. For each failure, record the test, model, column, and number of failing rows. 2. Take the failure that sits furthest upstream in the lineage. 3. Query a small sample of failing rows (LIMIT 20) to understand the pattern. 4. Trace the column upstream to the first model where the bad rows appear. Identify the cause: fan-out join, missing filter, missing deduplication, bad type cast, late-arriving data, or a real upstream data problem. 5. Fix the transformation SQL, run the build command for the model and its children, and re-run the tests. 6. Log: test, root cause, fix, failing rows before and after. Stop when: - All selected tests pass. Success. - You reach the max iteration count. - The cause is bad source data that the pipeline should not silently fix. Report it with sample keys. Rules: - Only build against the development target. Never run against production. - Do not delete, disable, or loosen tests (for example by adding a "where" clause or raising error thresholds) to make them pass. - Do not output personal data from samples. Show keys and counts. Final report: each failure, its root cause, the fix, and source data issues that need an owner.
When this loop fits
- dbt tests or data quality checks are failing after an upstream schema change.
- Duplicates or nulls showed up in a reporting table and you need to find where they start.
- You are adding tests to a pipeline and want the existing data cleaned up to pass them.
What happens on each pass
The agent repeats these steps until the stop condition holds or it runs out of iterations.
- Run the tests. Run the data tests for the selected models and list failures with their failing row counts.
- Trace upstream. Follow the failing column back through the lineage to the first model where bad rows appear.
- Fix the transformation. Correct the join, filter, deduplication, or type cast that introduces the problem.
- Rebuild and re-test. Rebuild the affected models in a development target and run the tests again.
MCP servers for this loop
The 6 strongest of 1306 matching servers, ranked by how closely they fit and then by trust score.
- 62Trust score 62 of 100, Good
Hosted BigQuery MCP server for Claude, ChatGPT and Cursor, with each action switched per account.
RegistryRemoteLicenseQualityMaintenanceLicense: A. MIT is a permissive open source license.Quality: Not graded. No README or tool list has been captured yet.Maintenance: A. Last commit or release 28 days ago. - 55Trust score 55 of 100, Fair
dbt model and column lineage, downstream impact and test proposals; no warehouse or dbt compile
RegistryLicenseQualityMaintenanceLicense: Not graded. No license is recorded for this server.Quality: Not graded. No README or tool list has been captured yet.Maintenance: Not graded. No commit or release date has been recorded. - 55Trust score 55 of 100, Fair
Predicts the DDL a dbt change will execute, before you run it. Reads files, not the warehouse.
RegistryLicenseQualityMaintenanceLicense: Not graded. No license is recorded for this server.Quality: Not graded. No README or tool list has been captured yet.Maintenance: Not graded. No commit or release date has been recorded. - 52Trust score 52 of 100, Fair
Data Quality Gate
Unknown author
Check entire lead, order and inventory CSV/JSONL against approved contracts before writes.
RegistryRemoteLicenseQualityMaintenanceLicense: Not graded. No license is recorded for this server.Quality: Not graded. No README or tool list has been captured yet.Maintenance: Not graded. No commit or release date has been recorded. - 52Trust score 52 of 100, Fair
64-bit Twitter Snowflake-compatible distributed unique ID generator with clock backwards drift prote
RegistryLicenseQualityMaintenanceLicense: Not graded. No license is recorded for this server.Quality: Not graded. No README or tool list has been captured yet.Maintenance: Not graded. No commit or release date has been recorded. - 51Trust score 51 of 100, Fair
bigquery-mcp
by snowleopard-ai · Python
A SnowLeopardAI-managed MCP server that provides access to Google BigQuery data.
RegistryLicenseQualityMaintenanceLicense: A. MIT is a permissive open source license.Quality: C. 60/100 for listing completeness. Lost points: no listed tools, short summary, no docs link.Maintenance: D. Last commit or release 7 months ago.
How to run this loop
- Connect the tools. Install a server the loop can use, for example BigQuery MCP by InsightfulPipe, or rely on your agent's built-in terminal and file access.
- Edit the settings block. Replace the example commands, targets, and limits with your project's own. Keep the iteration cap low on the first run.
- Paste it into an agent session. Start a fresh session in Claude Code, Cursor, or VS Code agent mode, in the repository root, and paste the prompt.
- Review before you merge. Read the final report and the diff. The loop leaves its work for you to review rather than shipping it.
Data Checks Until Clean questions
When does the Data Checks Until Clean loop stop?
It stops as soon as every data quality test on the selected models passes after a rebuild. It also stops after 8 iterations by default, or earlier when the prompt's own stop rules say it is stuck, and then reports what changed and what is left. Change the cap in the settings block at the top of the prompt.
Do I need dbt to run this loop?
No. Any tool that runs data tests and rebuilds models works: SQLMesh, Great Expectations, Soda, or plain SQL assertion queries in a script. Replace the two commands in the settings block and keep the rest of the prompt.
Why fix the most upstream failure first?
A single bad join near the source can fail tests in every model downstream of it. Fixing the earliest failure and rebuilding the children often clears several failures at once and avoids patching symptoms in reporting tables.