There's a question floating around that I keep seeing developers ask each other but never actually resolve: does the way you name your variables, functions, and types change what a coding agent produces?
Someone on V2EX put it well, pointing out that identifiers get embedded and run through attention, so a name written "for the machine" might tip the output one way or another. The only reply was the sensible-but-vague take that clear naming matters because code should be self-explanatory. True, but that's about humans reading code. It doesn't answer whether the agent's generation actually shifts.
I've wondered the same thing on real deliveries, and I got tired of guessing. So instead of arguing about it, I built a small harness to measure it. This is a debrief on how I set it up and what it's actually good for.
The problem with vibes-based naming debates
Every discussion about this collapses into anecdotes. "I renamed `data` to `invoiceLineItems` and the agent stopped hallucinating fields." Maybe. But you changed one thing, ran it once, and eyeballed the diff. That's not evidence, that's a story. The same thread on agent quality complaints elsewhere in the community shows how easily people misattribute behavior to the wrong cause, blaming a model when the real culprit is somewhere in the pipeline.
What I wanted was boring and repeatable: same prompt, same model, only the identifiers swapped between variants, results logged side by side so I could compare rows instead of trading impressions.
The harness, in plain terms
The shape is simple. You define one task prompt. You define a set of naming variants for the same underlying code skeleton: cryptic single letters, generic names like `data` and `handleThing`, and descriptive domain names. A script substitutes the identifiers into the prompt, sends each variant to a model, and writes every response to a table.
I put the whole thing on VicroCode because I didn't want to babysit a local environment for something I'd run in bursts. You can run Python online there, which covers the substitution logic, the model calls through the platform's Model Center APIs, and the logging. No machine setup, no dependency drift between the day I built it and the day I came back to re-run it.
Here's the core loop, trimmed to the idea:
import sqlite3, hashlib
VARIANTS = {
"cryptic": {"fn": "proc", "arg": "d"},
"generic": {"fn": "handleData", "arg": "data"},
"descriptive": {"fn": "normalizeInvoice", "arg": "rawInvoice"},
}
PROMPT_TEMPLATE = (
"Implement the function `{fn}` that takes `{arg}` and "
"returns a cleaned record. Only return code."
)
def run_variant(label, names, call_model):
prompt = PROMPT_TEMPLATE.format(**names)
output = call_model(prompt) # Model Center API call
digest = hashlib.sha256(output.encode()).hexdigest()[:12]
return {
"variant": label,
"prompt": prompt,
"output": output,
"output_hash": digest,
"output_len": len(output),
}
def log(rows, db="naming_experiment.db"):
con = sqlite3.connect(db)
con.execute("""CREATE TABLE IF NOT EXISTS runs(
id INTEGER PRIMARY KEY, variant TEXT, prompt TEXT,
output TEXT, output_hash TEXT, output_len INT, note TEXT)""")
con.executemany(
"INSERT INTO runs(variant,prompt,output,output_hash,output_len) "
"VALUES(:variant,:prompt,:output,:output_hash,:output_len)", rows)
con.commit(); con.close()The `output_hash` is the cheap trick that makes this worth doing. If the same variant produces the same hash across repeated runs, the model is stable for that input and any difference between variants is a real signal, not noise. If hashes scatter within a single variant, you learn the model is non-deterministic here and you need more runs per variant before you trust anything.
Why the results live in an editable table
The part that turned this from a toy into something I actually use is the storage. Every run lands in SQLite, and I review it through the SQLite editor rather than dumping logs to a console. That matters because reading agent output is a human judgment step. I sort by variant, read the outputs next to each other, and add a `note` on each row: did the descriptive variant include validation the cryptic one skipped?
Did the generic naming produce a more generic, less domain-aware implementation?
Editing rows in place beats re-running the whole batch every time I want to annotate. I can tag the rows I consider wins, filter to just those, and count them per variant. That's a comparison, not a hunch. When someone asks me later whether naming mattered on a given task, I point at the table.
One honest caveat: whether better names produce better agent output is exactly what this harness is meant to test, and I'm not going to claim a universal result here. That would be unverified. The point is that you generate your own evidence for your own model and task, instead of inheriting someone's anecdote.
What this is actually good for
Beyond settling the naming argument, the same skeleton is a general prompt-comparison rig. Swap the variant dimension from "identifier style" to "instruction phrasing" or "amount of context" and you've got a repeatable A/B harness for prompts. Given how much of current developer work assumes fluency with AI coding tools, having a way to measure prompt and naming choices instead of guessing is a genuinely useful habit. If you want to sharpen the fundamentals behind this, the AI coding material is a reasonable place to ground the practice.
A boundary worth naming: this harness compares models that are already available through the platform's Model Center. If the model you want to test isn't offered there, that comparison falls outside what I can wire up this way, and you'd be looking at a different setup entirely.
The takeaway from the debrief
The move that made the difference wasn't clever prompting. It was refusing to answer "does naming matter" from memory. Define the variants, hold everything else constant, log every output with a hash, and read the rows. Twenty minutes of setup replaces months of folklore. Whatever the answer turns out to be for your stack, you'll have earned it instead of guessed it.