Skip to content
Journal

3 min readBuildingUpdated

Where AI Belongs in a Data Quality Workflow

Letting a model judge the data is the obvious approach. QualityPlus keeps it away from the score: deterministic checks produce the score, AI assists with the findings, and a person makes the final call.

The temptation to put AI everywhere

QualityPlus started from a familiar problem. Operational datasets arrive incomplete, duplicated or inconsistent, and the fixes tend to happen by hand in a copy of a spreadsheet, with no record of what changed or who agreed to it.

One obvious approach would have been to hand the whole problem to a model. The trouble appears as soon as you think about the second run. Would the same dataset get the same score tomorrow? Could anyone explain why a row failed? If a value changed, who changed it, the model or a person? For a quality check, those questions are the whole point.

Some checks should stay deterministic

So QualityPlus is built around a boundary. Anything that produces the score stays deterministic: explicit rules, run the same way every time, explainable row by row. It scores four dimensions: completeness, uniqueness, validity, and consistency with something already trusted.

A missing value, a duplicate key or a value outside a defined range can be evaluated directly against explicit rules, and a rule does that more cheaply, more quickly and more predictably than a model would. The failures are kept too. The failed cells become the work, and the score becomes the start of a workflow instead of the end of a report.

Fig. 1 / Where the AI sits

Deterministic

  1. 01Data

    A dataset uploaded into a project.

  2. 02Quality assessment

    Completeness, uniqueness, validity and consistency rules, run the same way every time.

  3. 03Findings

    A score per dimension and the exact rows and cells that failed.

AI assistance

  1. 04AI assistance

    Suggests rules, summarises findings and proposes fixes with a reason and a confidence.

Human review

  1. 05Human review

    Accept, modify or reject each suggestion. Owners decide, and a rejection needs a reason.

  2. 06Approved result

    A corrected version, with a record of how every value changed.

The boundary in QualityPlus. Scoring and findings are deterministic. AI assistance sits between the findings and the person reviewing them, and it never writes the score or changes a value on its own.

Where AI actually helped

There are three places, each a separate n8n workflow that calls a model through Ollama. One suggests validity rules from column names and an oil and gas domain knowledge base. One summarises a saved score. One proposes fixes for failed cells, each with a reason and a confidence, and leaves out cells it cannot fix with confidence. Every output is a suggestion a person can accept, change or ignore.

All three are optional. If a workflow is not configured, its feature is hidden and the quality workflow still runs end to end. If switching the AI off would break the quality check, the AI has ended up somewhere it should not be.

Fig. 2 / Try it: switch the AI off

Well IDUniqueStatusCompleteLift typeIn reference listDepth (m)Range 0 to 6,000
W-101ACTIVEESP2,140
W-102ACTIVEGL1,880
W-102SUSPENDEDESP2,010
W-104blankGL1,960
W-105ACTIVEESPP-420

DeterministicRules run the same way every time. 16 of 20 cells pass: 80%.

Illustrative sample data · not a QualityPlus capture

Step through the workflow, then switch the AI off and run it again. The score, the findings and the review still work; only the suggestions disappear.

Humans still need the last word

Every correction records how it was made: entered manually, accepted from an AI suggestion, or accepted and then modified. Both the AI’s value and the final value are kept. Owners, co-owners and editors can submit corrections. Only owners and co-owners approve or reject them, and a rejection needs a reason. Rule templates are reviewed and approved before other projects reuse them.

Keeping the AI’s answer beside the human’s made the system auditable. It also makes the AI measurable, which is a better test than how convincing its suggestions sound.

The workflow mattered more than the model

The workflow phase is computed from the counts of failed, addressed and pending cells, so it cannot drift away from the data. Each AI touchpoint is a separate webhook, so one can change without touching the scoring. An early Python backend on Render was replaced by talking to Supabase directly, which left less to break.

The parts that made QualityPlus usable were the rules, the record of every decision and the approval step. The same principle sits in FORMA: AI can assist, but it should never be the hidden execution engine.

What I would change next

QualityPlus is a proof of concept. I demonstrated it at the SPE AI Summit, but what I know about it comes from building and demonstrating it, not from long-term use.

  • Use the record of corrections to evaluate the fix suggestions properly, column type by column type.
  • Make consistency checks understand relationships between datasets, beyond matching values against a single reference.
  • Test the boundary with more reviewers, to see whether they read the AI’s reasoning or only its suggested value.