The situation
An intelligence team needed a landscape catalog: a spreadsheet mapping a specific online ecosystem, with evidence behind every row. The work ran as manual research batches. Someone had to trigger each sweep, run the searches, capture screenshots by hand, and paste them into the deliverable. Each batch produced a handful of rows, days apart. Scope rules lived in a prose brief that nothing enforced.
What I did
I built an unattended daily pipeline. Every morning it searches a rotating set of sources, deduplicates against everything already found, captures forensic-grade evidence for each candidate, gets exactly one AI judgment call per candidate, validates the result against the client's own schema, appends it to the deliverable with the screenshot embedded in-cell, and posts the day's result to Slack.
The discipline is in what it refuses:
- A capture that hit a bot-wall or rendered blank is automatically classified unusable and cannot be cited. Previously, ten bad captures sat marked complete for 20 days before a human noticed.
- Scope rules moved from prose into code: a machine-readable scope file, runtime gates on every row, and a ratchet test that blocks any new out-of-scope row forever.
- Every write is snapshotted first. A failed validation restores the deliverable byte-identical and raises an alert.
One measured repair along the way: the deduplication was silently discarding about a third of live search results by blocking entire hosts on the strength of a single old row. Fixing it raised fresh candidates per run by nearly three quarters.
The number
The deliverable grew from 113 rows to 174, each with its screenshot embedded and its evidence archived with a cryptographic hash. Best single day: 18 new validated rows, unattended. The human role shifted from running searches to reviewing a finished morning result.
Anonymized: confidential client and subject matter. Every number is from the engagement's own logs.
If your team produces a deliverable by hand that a system could produce every day, the assessment week finds out whether the same shape of system fits, and what it would cost.
Book a 30-minute callMore work: the analyst that refuses to guess · the pipeline that failed 95 times silently · the investigations OS