There's a specific failure mode that shows up once a team leans hard on AI to write code. A developer on V2EX laid out the symptoms plainly: they're losing the muscle to write code by hand, the AI's output is verbose and not always clean, and they feel a shrinking grip on whether the project is actually correct. The replies that landed hardest weren't about better prompts. They were about judgment and verification becoming the real job. One put it bluntly: the skill now is building a trustworthy verification mechanism so nothing big slips through, and when it does, you can catch and roll it back fast.
That same thread has a quieter truth buried in it. When the AI produces version after version faster than you can read them, line-by-line human review stops scaling. So teams bolt on AI reviewers. And that's where a second problem starts.
The noise problem nobody warns you about
The clearest writeup of this came from the team behind HeyDeer, an AI code-review tool. Their engineers were heavy Claude and Codex users, PR volume climbed, and the old rule of carefully reviewing and running every line became impossible. They tried existing reviewers and hit the same wall with each: too many false positives. Engineers ended up spending more time sorting trivial flags than shipping, and PRs dragged.
The part worth stealing is the failure they describe next. When you hand those raw findings to a coding agent and tell it to fix them, the agent cheerfully fixes the false positives too. The codebase swells with changes that were never real problems. You've automated the production of noise and then automated acting on it.
Their fix was a verifier step: for every issue the AI surfaces, estimate its ROI and find a way to reproduce it before it goes anywhere. Approve simple PRs outright, hold complex ones, and only let an autofix loop run on findings that survived the gate. That one idea, a verifier that demands reproduction and scores value before a human or a fix-agent ever sees the item, is durable and portable. You don't need their whole product to use it.
What this gate is, and what it isn't
I want to be exact about scope, because it's easy to oversell. The thing we're building does not scan a repository, does not run on a PR, and does not touch CI. It is not a reviewer. It assumes findings already exist, from whatever reviewer or model you point at your code, and it does one job: order and filter those findings. It holds each one in a ledger, tracks whether it's been reproduced, scores its ROI, and refuses to promote anything that hasn't earned the trip to a human or an autofix run.
Treat it as a triage gate between "the AI said something" and "someone or something acts on it." That boundary is the whole point. HeyDeer's claim that this reduces false positives is theirs; I'm treating any false-positive-rate improvement as unverified. What I can defend is the workflow logic: unreproduced findings shouldn't drive edits.
Building the gate on VicroCode
Three pieces, all inside the platform.
First, a findings ledger in SQLite. Each finding gets a row: an id, the file and line it points at, the rule or category, the reviewer's raw message, a severity, a status of `unreproduced` or `reproduced`, a reproduction note, and a computed ROI score. The status field is the spine of the whole thing. A finding enters as `unreproduced` and stays quarantined there until someone or something attaches a concrete way to trigger it. You can edit and inspect rows directly with the in-platform SQLite editor, which matters more than it sounds, because triage is messy and you'll want to correct statuses and notes by hand while you tune the rules.
Second, a scoring and intake endpoint in Python. Host a small service that accepts findings as JSON, writes them to the ledger, and computes an ROI score on the way in. Keep the scoring boring and explicit so you can defend it: weight by severity, down-weight categories that historically produce noise for you (cosmetic copy nits, style churn), and multiply by a reproduction factor so anything still `unreproduced` is pinned low no matter how scary its message sounds. The logic is a few functions you can run Python online without standing up any infrastructure of your own. A second route flips a finding to `reproduced` once a repro note is attached, which is the only thing that lets its real score surface.
The design decision that makes this work: promotion is gated on status, not on severity. A `critical` finding with no reproduction sits below a `minor` finding someone actually triggered. That inversion is what stops a fix-agent from charging at confident-sounding noise.
Third, an HTML triage board, published and hosted on the platform. Three columns map to the lifecycle: Unreproduced, Reproduced, and Promoted. The board reads the ledger through the endpoint and lets a human do the one judgment call that can't be automated away, deciding whether a flagged thing is worth reproducing at all. Drag a card, attach a repro note, and only then does it become eligible to leave. Findings that nobody will reproduce age out visibly instead of silently becoming edits.
Wiring it into a real loop
In practice you run it like this. Your AI reviewer, whatever it is, posts its findings to the intake endpoint. Everything lands as `unreproduced` and low-scored. A person skims the board, which is now a ranked queue instead of a wall of equally-urgent flags, and spends attention only on items worth a reproduction attempt. Reproduced items climb the score and move to Promoted. Your fix-agent, if you use one, is told to pull only from Promoted. That single constraint is the difference between an autofix loop that tightens the codebase and one that bloats it.
If you want the gate to learn your project's quirks, encode them in the scoring weights rather than in prompts. HeyDeer did this with check files tuned to their stack. You'd do the equivalent by adjusting category weights: if your repo has house rules that make certain patterns fine, push those categories down so they rarely clear the gate. The knobs live in your Python, in plain sight, which is exactly where you want verification logic you're asked to trust.
Where the edges are
Be honest with yourself about what sits outside this. The gate doesn't fetch code, doesn't understand your diff, and doesn't decide whether a finding is correct, it decides whether it's reproduced and worth acting on. Reproduction itself still happens wherever your code runs; the ledger only records that it happened. And the ROI weights are a model of your priorities, not a measurement, so expect to tune them for a while before the ranking feels right.
What you get in return is small and concrete. The volume problem from that V2EX thread, versions arriving faster than you can read them, becomes a ranked queue instead of an undifferentiated pile. The judgment that the commenters said now defines the job gets a surface to live on. And the specific trap HeyDeer named, agents dutifully fixing things that were never broken, gets a gate in front of it that costs an afternoon to build. If you're newer to shaping these review workflows, the platform's AI coding material is a reasonable place to get oriented before you start tuning weights.
None of this proves your false-positive rate drops. It does mean no unreproduced finding ever reaches a fix-agent, and that constraint alone changes the shape of the problem.