3 min readBuildingUpdated
Where AI Belongs in a Data Quality Workflow
Letting a model judge the data is the obvious approach. QualityPlus keeps it away from the score: deterministic checks produce the score, AI assists with the findings, and a person makes the final call.
The temptation to put AI everywhere
QualityPlus started from a familiar problem. Operational datasets arrive incomplete, duplicated or inconsistent, and the fixes tend to happen by hand in a copy of a spreadsheet, with no record of what changed or who agreed to it.
One obvious approach would have been to hand the whole problem to a model. The trouble appears as soon as you think about the second run. Would the same dataset get the same score tomorrow? Could anyone explain why a row failed? If a value changed, who changed it, the model or a person? For a quality check, those questions are the whole point.
Some checks should stay deterministic
So QualityPlus is built around a boundary. Anything that produces the score stays deterministic: explicit rules, run the same way every time, explainable row by row. It scores four dimensions: completeness, uniqueness, validity, and consistency with something already trusted.
A missing value, a duplicate key or a value outside a defined range can be evaluated directly against explicit rules, and a rule does that more cheaply, more quickly and more predictably than a model would. The failures are kept too. The failed cells become the work, and the score becomes the start of a workflow instead of the end of a report.
Fig. 1 / Where the AI sits
Deterministic
01Data
A dataset uploaded into a project.
02Quality assessment
Completeness, uniqueness, validity and consistency rules, run the same way every time.
03Findings
A score per dimension and the exact rows and cells that failed.
AI assistance
04AI assistance
Suggests rules, summarises findings and proposes fixes with a reason and a confidence.
Human review
05Human review
Accept, modify or reject each suggestion. Owners decide, and a rejection needs a reason.
06Approved result
A corrected version, with a record of how every value changed.
Where AI actually helped
There are three places, each a separate n8n workflow that calls a model through Ollama. One suggests validity rules from column names and an oil and gas domain knowledge base. One summarises a saved score. One proposes fixes for failed cells, each with a reason and a confidence, and leaves out cells it cannot fix with confidence. Every output is a suggestion a person can accept, change or ignore.
All three are optional. If a workflow is not configured, its feature is hidden and the quality workflow still runs end to end. If switching the AI off would break the quality check, the AI has ended up somewhere it should not be.
Fig. 2 / Try it: switch the AI off
| Well IDUnique | StatusComplete | Lift typeIn reference list | Depth (m)Range 0 to 6,000 |
|---|---|---|---|
| W-101 | ACTIVE | ESP | 2,140 |
| W-102 | ACTIVE | GL | 1,880 |
| W-102 | SUSPENDED | ESP | 2,010 |
| W-104 | blank | GL | 1,960 |
| W-105 | ACTIVE | ESPP | -420 |
DeterministicRules run the same way every time. 16 of 20 cells pass: 80%.
Illustrative sample data · not a QualityPlus capture
Humans still need the last word
Every correction records how it was made: entered manually, accepted from an AI suggestion, or accepted and then modified. Both the AI’s value and the final value are kept. Owners, co-owners and editors can submit corrections. Only owners and co-owners approve or reject them, and a rejection needs a reason. Rule templates are reviewed and approved before other projects reuse them.
Keeping the AI’s answer beside the human’s made the system auditable. It also makes the AI measurable, which is a better test than how convincing its suggestions sound.
The workflow mattered more than the model
The workflow phase is computed from the counts of failed, addressed and pending cells, so it cannot drift away from the data. Each AI touchpoint is a separate webhook, so one can change without touching the scoring. An early Python backend on Render was replaced by talking to Supabase directly, which left less to break.
The parts that made QualityPlus usable were the rules, the record of every decision and the approval step. The same principle sits in FORMA: AI can assist, but it should never be the hidden execution engine.
What I would change next
QualityPlus is a proof of concept. I demonstrated it at the SPE AI Summit, but what I know about it comes from building and demonstrating it, not from long-term use.
- Use the record of corrections to evaluate the fix suggestions properly, column type by column type.
- Make consistency checks understand relationships between datasets, beyond matching values against a single reference.
- Test the boundary with more reviewers, to see whether they read the AI’s reasoning or only its suggested value.
