A builder shared a Dovetune post-mortem that's worth sitting with, because it names two failure modes most of us learn the expensive way. The site turns a written memory into an original song: write a story, polish it, generate editable lyrics, then generate the full track. Full preview is free, and you pay $19.99 to take the song home. One person, aimed at the US gifting market. Nothing exotic in the stack. But the two things that broke are the two things that break almost every AI pipeline once real traffic shows up.
The two traps, stated plainly
The first was mistaking an account rate limit for a model outage. Claude returned 429, so the natural reflex was to switch to Gemini. Still 429. Both requests were going through the same reseller account, and that account shared a single rate-limit pool across models. Buried in the docs were two lines that changed everything: limits are counted per account, and rejected requests don't enter a queue. So while the account is throttled, switching models does nothing, and an immediate retry doesn't get your request to wait in line either. The "fail, then swap model" fallback looked like resilience but never distinguished *why* a call failed. For that provider, an account-level throttle means you have to switch providers to keep going; swapping models inside the same provider only makes sense when the upstream returns a 500.
The second trap was a synchronous wait hiding inside an async flow. Full song generation takes about 180 seconds, so that step was correctly built as an async job with polling or callbacks. The lyric step in front of it, though, was a synchronous call with a 15-second timeout. It got treated as a minor intermediate step while all the attention went to the slower song stage. In practice, the moment lyrics crossed 15 seconds, the entire flow had already failed, and the careful async design behind it never got a chance to run. Worse, lyrics sit *before* payment, so free users hit that step over and over.
The real lesson isn't "raise the timeout"
The thing I keep coming back to: a user experiences the whole pipeline, not individual call styles. They don't know or care which step was synchronous and which polled a job. So when you audit a slow, multi-stage task, you can't just bump a timeout number and move on. You have to lay out, for every single step, three things: how it waits (synchronous call vs async job), what its timeout is, and where a failure sends the request next. The lyric step deserved that scrutiny precisely because it looked trivial, "it only generates a few lines of text," and that assumption is what let a 15-second ceiling quietly gate the paid flow.
That's a reusable artifact, not a one-off fix. So instead of carrying it around in my head, I'd make it a living record the pipeline writes to itself.
Building the ledger as a hosted Python service
Here's what I'd stand up. Wrap each stage of the pipeline in a small Python harness and run Python online so the orchestration lives next to the record it produces. Every step emits one row before and after it executes: step name, declared wait mode (`sync` or `async_job`), configured timeout, actual elapsed time, final status, and the failure destination, meaning where control went when it failed (retry same provider, switch provider, surface error to user, or dead-end).
The schema is deliberately boring:
CREATE TABLE step_log (
id INTEGER PRIMARY KEY AUTOINCREMENT,
run_id TEXT NOT NULL,
step_name TEXT NOT NULL,
wait_mode TEXT NOT NULL, -- 'sync' | 'async_job'
timeout_s INTEGER,
elapsed_s REAL,
status TEXT NOT NULL, -- 'ok' | 'timeout' | 'rate_limited' | 'upstream_5xx' | 'error'
failure_route TEXT, -- 'retry_same' | 'switch_provider' | 'user_error' | 'dead_end'
provider_account TEXT, -- so you can see shared pools
created_at TEXT DEFAULT (datetime('now'))
);That `provider_account` column is doing quiet but important work. If Claude and Gemini calls both log the same account value, the shared rate-limit pool becomes visible in the data instead of being a surprise at 2am. A 429 row with `status='rate_limited'` and `failure_route='switch_provider'` tells you instantly whether your fallback is pointed at a different account or just spinning on the same throttled one.
Because this is a plain SQLite file, I keep an SQLite editor open alongside it to inspect runs, correct a mislabeled wait mode, or query which step contributes the most timeouts. Something as simple as grouping by `step_name` where `status='timeout'` would have put the lyric step at the top of the list on day one.
A status board you can actually read
The ledger is only useful if you look at it, so I'd pair it with a small HTML status board and use web app hosting to keep it reachable. Nothing fancy: a table per recent run showing each step as a row, color-coded by status, with the declared timeout next to the actual elapsed time. The gap between those two numbers is where the lyric problem lived in plain sight. A second view aggregates across runs so you can see, over a day of free-tier traffic, which pre-payment step burns the most time and triggers the most retries.
The board answers the question the debrief was really asking: for a long task, where does each step wait, how long is it allowed to wait, and where does it go when it gives up. Once that's a dashboard instead of a hunch, "it only generates a few lines of text" stops being a safe assumption.
The boundary, said honestly
Here's what this build does *not* do, and it matters. VicroCode can host the Python orchestration, store the ledger, and serve the board, but it cannot change how an upstream reseller counts its limits. If a provider scopes throttling to the account and refuses to queue rejected requests, that's their rule, and no amount of logging on my side rewrites it. The reseller landscape around this is noisy, discounted model access, account bans, "求上游对接" posts, and the ledger doesn't make any of that more stable. What it gives you is clear-eyed evidence: it shows *that* you're hitting an account-scoped pool and *where* your fallback misfires, so you can make the real decision, which is usually switching providers or re-sequencing the flow, with data instead of guesswork.
The fix for the Dovetune author's two traps was never a bigger timeout or a cleverer retry. It was making the pipeline describe its own waiting out loud. That's a few hours of work and a boring SQLite table, and it's the kind of thing you only build once but check constantly.