There's a lot of noise in the agent space right now, and most of it is about speed, cost, or which model is cheapest to route through. But buried in a weekend-project writeup about a Mac computer-use agent called DeskMind was one idea worth stealing. The author trained two small local models to drive the desktop, and mentioned almost in passing that when a task has two valid readings, the agent asks first. The example they gave: if the table has two orders under the name Lisa Wong, it asks which one you mean instead of picking one and writing it in.
That's the whole thing. Not the fine-tuning, not the local inference, not the MLX setup. The durable idea is a gate: before an agent commits to an action, check whether the input it's working from is actually unambiguous. If it isn't, stop and resolve it with a human rather than guessing and moving on.
Why this is the part that matters
Most agent failures that cause real damage aren't dramatic. They're quiet. The agent reads "update Lisa Wong's shipping address," finds two matching records, and silently edits the first one. Nobody notices until the wrong customer gets someone else's package. The agent reported success. The log says it finished. By the DeskMind author's own account, their published build never claimed completion on an unfinished task, though they're clear that the task set was small, self-authored, and the pass rate is self-reported, so treat that number as unverified.
The interesting failure mode they explicitly asked testers to hunt for was the one where it "should have asked but guessed instead." That framing is the useful part. You can't eliminate ambiguity in the input, but you can detect it and refuse to proceed until it's resolved. That's a design decision, not a model capability, which means you can build it yourself without training anything.
Rebuilding the gate as a hosted endpoint
Here's what I'd actually build, and it maps cleanly onto what you can host without standing up your own infrastructure. The gate is a small Python service that sits between the agent and the record store. The agent doesn't get to touch records directly. It calls the gate, the gate decides whether the match is clean, and only a clean match comes back as actionable.
The record store itself fits a SQLite database well: it's a single file, it's queryable, and you can hand-edit it when you need to seed test data or fix a bad row. You can keep the whole thing on VicroCode and lean on its run Python online support to execute the matching logic, so there's no separate server to babysit.
The core check is boring on purpose:
import sqlite3
import json
def resolve(field, value, db_path="records.db"):
conn = sqlite3.connect(db_path)
conn.row_factory = sqlite3.Row
cur = conn.execute(
"SELECT id, name, email, order_ref FROM orders WHERE name = ?",
(value,),
)
rows = [dict(r) for r in cur.fetchall()]
conn.close()
if len(rows) == 0:
return {"verdict": "not_found", "candidates": []}
if len(rows) == 1:
return {"verdict": "resolved", "candidates": rows}
return {"verdict": "needs_clarification", "candidates": rows}Notice there are three outcomes, not two. "Resolved" is the only one the agent is allowed to act on. "Not found" and "needs clarification" both block, for different reasons. The agent has to come back with a disambiguating key, an order reference, an email, a record id, before it gets a green light. Parameterized queries here aren't optional; the value is coming from an agent interpreting free text, so you treat it as untrusted input and bind it rather than concatenate it.
Logging the human choice is the whole point
When the verdict is "needs clarification," something has to resolve it, and that something should usually be a person the first many times. The step people skip is writing that choice down. Every time a human picks "the Lisa Wong with order ref 4471," you log the ambiguous input, the candidate set, and the resolution:
def log_choice(raw_input, candidates, chosen_id, db_path="records.db"):
conn = sqlite3.connect(db_path)
conn.execute(
"INSERT INTO disambiguation_log "
"(raw_input, candidate_ids, chosen_id, ts) "
"VALUES (?, ?, ?, datetime('now'))",
(raw_input, json.dumps([c["id"] for c in candidates]), chosen_id),
)
conn.commit()
conn.close()That log is where the value accrues. After a few weeks you can look at it and see which inputs are chronically ambiguous, and often fix the data or the upstream prompt so the collision stops happening. A built-in SQLite editor makes that review loop fast: you query the log, spot the pattern, and patch the record store in the same place without exporting anything.
Exposing it so the agent actually blocks
A gate the agent can ignore isn't a gate. The way to make it stick is to expose the resolver as the only sanctioned path to a record, published as an endpoint the agent calls as a tool. You host the Python resolver as an API endpoint and register it as an in-platform tool call, so from the agent's side, resolving a record is a function it invokes, and a `needs_clarification` verdict is a hard stop it has to handle before any write.
This is squarely in AI agent development territory: the behavior you want isn't smarter inference, it's a tool contract that makes guessing impossible by construction. The agent can be as eager as it likes; if the gate says the input matches two records, there's no action to take until the ambiguity is gone.
Where the boundary sits
Two honest limits worth naming. First, DeskMind runs local small models on Apple silicon and drives the actual desktop; that computer-use layer is not something you'd reproduce here, and nothing in the confirmed capability set does OS-level automation. What ports cleanly is the decision pattern, not the agent's hands. Second, this gate only disambiguates against records you can structure and query. The DeskMind author was candid that genuinely vague instructions like "organize this folder" are a different, harder problem, and a record-matching gate does nothing for that. It solves the specific, common case where the input is clear but points at more than one row.
That narrowness is a feature. You can build this in an afternoon, seed a SQLite table, host the Python check, wire it as a tool, and you've closed off a whole category of quiet, expensive mistakes. The agent stops being confidently wrong and starts asking the one question a human would have asked anyway.