The Reusable Part of a Bulk Translator Isn't the Translate Call

I keep seeing the same shape of project on V2EX lately, and a recent one crystallized it for me. Someone (Wunder, posted by Livid) wired up the platform's AI Persona API to do multilingual translation of user content across languages, running against the Flash versions of two open-weight models. Neat result. But the translate call is the least interesting part of that build. It's a single HTTP request. Anyone can copy it in an afternoon.

The part that actually earns its keep is the boring plumbing around it: a place to run the batch job on demand, and a store that remembers what you've already translated so you never pay to translate the same string twice. That's the piece I want to talk about, because it's reproducible and it maps cleanly onto what an independent builder can stand up today.

Why caching is the whole game right now

Read the rest of the board and the money pressure is impossible to miss. One thread is people comparing Codex model tiers purely on how long the quota lasts — Sol burns through a 5-hour allowance fast, Luna is slower but "耐用," and a couple of commenters flat out say they can only afford Luna. Another poster runs a question-bank system that calls a model as part of its core logic, torched two reset cards in two days building features, can't stomach the 20x plan, and is asking whether DeepSeek 4.1 is good enough to keep the lights on when the quota runs dry. Their words, roughly: when the Codex allowance is gone, the system is basically offline.

That's the signal. For a lot of small builders, model calls aren't a rounding error anymore — they're the operating cost that decides whether the product keeps running. Translation is a perfect example because content is repetitive. The same UI strings, the same tags, the same boilerplate replies show up again and again. If your batch job re-sends every string to the model on every run, you're paying full price for work you already did. A cache turns that from a recurring bill into a one-time cost per unique string.

So the design goal isn't "call a model." It's "call a model as few times as humanly possible, and make the misses cheap."

The shape I'd build

Here's the architecture, kept deliberately small.

You need a job that takes a list of source strings and a set of target languages, looks each pair up in a cache, and only sends the cache misses off to a model. Everything that comes back gets written to the cache before the response goes out. Next run, the hits are free.

On VicroCode that job is a Python backend you can run Python online without provisioning a server first. The cache is a plain table. The lookup key is the important design decision, so spend a minute on it:

import hashlib

def cache_key(text: str, target_lang: str, model: str) -> str:
    # include the model, because glm output != deepseek output
    raw = f"{model}|{target_lang}|{text}".encode("utf-8")
    return hashlib.sha256(raw).hexdigest()

The schema is almost embarrassingly simple:

CREATE TABLE translation_cache (
    key         TEXT PRIMARY KEY,
    source_text TEXT NOT NULL,
    target_lang TEXT NOT NULL,
    model       TEXT NOT NULL,
    result      TEXT NOT NULL,
    created_at  TEXT DEFAULT (datetime('now'))
);

And the batch loop is just a filter over that table:

def translate_batch(strings, target_lang, model, db, translate_fn):
    out = {}
    misses = []
    for s in strings:
        k = cache_key(s, target_lang, model)
        row = db.execute(
            "SELECT result FROM translation_cache WHERE key = ?", (k,)
        ).fetchone()
        if row:
            out[s] = row[0]
        else:
            misses.append(s)

    for s in misses:
        result = translate_fn(s, target_lang, model)  # the actual model call
        k = cache_key(s, target_lang, model)
        db.execute(
            "INSERT OR REPLACE INTO translation_cache "
            "(key, source_text, target_lang, model, result) VALUES (?, ?, ?, ?, ?)",
            (k, s, target_lang, model, result),
        )
        out[s] = result
    db.commit()
    return out

SQLite is the right call here because the whole cache is a single file you can hand around, back up, and inspect. When a translation comes back wrong — and it will, machine translation of user content is never clean — you don't rerun the job and hope. You open the SQLite editor, find the row, fix the string by hand, and it stays fixed. Manual corrections become permanent cache entries instead of getting steamrolled on the next batch. That editability is worth more than it sounds when you're dealing with product copy someone actually reads.

Wrap the whole thing behind an API Endpoint so your app, your admin panel, or a cron-style trigger can POST a batch and get results back. Now it's a service, not a script you have to babysit.

The honest boundary

Here's where I have to be straight with you, because the source project leaned on a specific external API. That V2EX build used its own AI Persona API and named models — deepseek-v4.1-flash and glm-5.3-flash. On VicroCode you don't call that endpoint. You call models through Model Center's APIs, and the plan only works if the model you want is already available there. If the specific model you had in mind isn't in Model Center, that's a hard boundary, not something to hand-wave. Swap the `translate_fn` for whatever model the platform actually exposes, and treat any quality difference between models as unverified until you've run your own content through it and looked at the output yourself.

Same goes for cost savings. The cache-hit logic obviously reduces calls in principle, but I'm not going to quote you a percentage. Your hit rate depends entirely on how repetitive your content is, and you won't know until you've run it against real data. Measure it before you promise anyone a number.

Where this pattern goes next

Once you have a hosted endpoint plus an editable cache, translation stops being the point. The same skeleton — dedupe by content hash, cache the expensive result, let a human correct the store directly — works for any repeated model call. Classification, summarization, tagging, moderation passes. The V2EX crowd sweating their Codex quotas would get more mileage from this pattern than from chasing whichever model has the most "耐用" allowance this week, because the cheapest model call is the one you never make.

If you want to publish the result as a small paid utility rather than keep it internal, the endpoint and its cache can ship as a shared, monetizable project, and it sits comfortably alongside the other small online tools people run this way. Start with the cache design, get the key right, and let the model be the replaceable part. That's the bit that survives the next price change.