ASK Consulting builds AI systems that hold up under hostile review. This one passed the most hostile review there is: the client's own engineers merging it into production.
The situation
A market research platform had the fraud problem every survey platform has: bad actors polluting the data buyers pay for. Their quality-enforcement team investigated flagged tickets by hand, an analyst digging through MongoDB queries per ticket. Their verdict on early tooling attempts was blunt: exploratory analysis where they needed deep-dives on the flagged question.
What I did
I built an AI quality-enforcement agent that sits on their own data. It reads a flagged ticket, pulls the evidence rows before any AI judgment happens, and walks a 5-step human-in-the-loop flow to a buyer-facing report card.
The part that made it trustworthy is not the AI. It is the gates around it:
- Missing evidence cannot render a "clear" verdict.
- Numeric claims are validated against the actual query counts.
- Contradictory device evidence blocks the report instead of shipping inside it.
Alongside it: a weekly threat intelligence report (one week's run read 8,763 messages across 19 fraud-community channels), feeding detection rules and an executive brief their leadership reads.
The number
The finished delivery was reviewed and merged into their production codebase by their own engineers: 369 files were re-applied onto their own diverged codebase, with 18 conflicts resolved by hand, and a later merge landed with zero conflicts and 2,498 tests passing. The engagement is monthly and has run six months and counting.
Anonymized at the founder's direction. Every number is from the engagement record.
If your AI feature is doing something like this, the assessment week finds where the same kind of checks belong. Start with one thing: one week, findings and priced options by Friday.
Book a 30-minute callMore work: the pipeline that failed 95 times silently · the investigations OS · the research desk that runs itself