A reading app made the rounds this week. The pitch was simple enough: the whole of *The Little Prince*, 27 chapters, re-translated from the French, Chinese and English side by side, with bilingual narration on every paragraph and the text lighting up word by word as the voice reads. Add 119 illustrations and an opening animation and you get something closer to a talking picture book than a textbook. The builder said he'd gone a full 24 hours without sleep answering comments and patching bugs, and that the project had crossed 1,400 GitHub stars in under a day. One commenter reported the site was down; another said it loaded fine. Normal launch-day chaos.
I kept staring at one detail: the words lighting up on cue. Everyone fixates on the catalog — which books, whose voice, how pretty the art is. But that stuff is perishable. You'll swap books. You'll re-record voices. The illustrations will get redrawn. The thing that actually survives all of that churn is the alignment: a mapping from each spoken word to the millisecond it starts and ends. Get that right and store it cleanly, and the content on top becomes replaceable. Get it tangled up inside your rendering code and you're rebuilding the hard part every time the content changes.
So as a debrief-to-a-peer exercise, I wanted to rebuild only the durable half — the timing engine — and leave the books and voices as swappable inputs. Three pieces: a table of word timings, an endpoint that serves them, and a player that reads them.
Why treat the timing data as a table?
The whole trick is refusing to store timing as a blob glued to one specific audio file. It's rows. Each word is a row with a track it belongs to, its position in the sentence, the word text itself, and a start and end time in milliseconds.
CREATE TABLE word_timings (
id INTEGER PRIMARY KEY,
track_id TEXT NOT NULL,
word_index INTEGER NOT NULL,
word TEXT NOT NULL,
start_ms INTEGER NOT NULL,
end_ms INTEGER NOT NULL
);
CREATE INDEX idx_track ON word_timings(track_id, word_index);That's it. A paragraph of narration is just one `track_id` with its words ordered by `word_index`. When you want to add a new chapter, you insert rows — you don't touch the player. When a recording gets redone and the timing shifts, you update `start_ms` and `end_ms` for that track and nothing downstream knows the difference.
This is where a hosted SQLite database earns its keep: it's a real relational store, not a flat file you have to re-parse on every request, and you can poke at the rows directly. Early on I spent a lot of time eyeballing suspicious timings — a word that claims to end before it starts, two words that overlap, a gap that's suspiciously long. Being able to open the table in a browser-based SQLite editor and fix a bad row by hand, instead of writing a one-off script every time, saved me more hours than I'd like to admit.
One honest boundary: *producing* those millisecond timings in the first place is the genuinely hard part, and it's the part that lives outside what I'd lean on the platform for. Forced alignment from an audio file to a transcript needs a specific model, and I won't pretend one is sitting ready to call. The confirmed Model Center route only helps for models already available on the platform, so assume the alignment itself is generated elsewhere or hand-corrected, and that the platform's job starts once you have the numbers. Whether any particular alignment model is reachable is unverified — don't design around it until you've checked.
What does the Python endpoint actually do?
The backend has one job: given a track, return its words in order. No templating, no rendering, no business logic creeping in. A small handler over the SQLite file does it, and keeping the query parameterized means a track id from the URL can't turn into an injection.
import sqlite3
def handler(request):
track_id = request.query.get("track_id", "")
conn = sqlite3.connect("reader.db")
conn.row_factory = sqlite3.Row
rows = conn.execute(
"SELECT word_index, word, start_ms, end_ms "
"FROM word_timings WHERE track_id = ? "
"ORDER BY word_index",
(track_id,),
).fetchall()
conn.close()
return {"track_id": track_id, "words": [dict(r) for r in rows]}Because this only serves public reading content and never writes, I didn't put auth in front of it — worth saying out loud, since a network endpoint without access controls is a decision, not a default. If you ever add anything that writes back (bookmarks, progress, user notes), that changes and you'd want a real gate. For a read-only timing service, open is fine. Being able to run Python online and expose it as a hosted endpoint means this handler goes from a file to a live URL without me standing up a server or wiring a deploy pipeline.
Why keep the player dumb?
The front end fetches the word list once, renders each word as a span, then listens to the audio clock and flips the highlight. The lookup is a binary search over start times so it stays cheap even on long paragraphs — you don't want to scan every word on every `timeupdate` tick.
<audio id="a" src="track1.mp3" controls></audio>
<p id="text"></p>
<script>
let words = [];
async function load(trackId) {
const res = await fetch(`/timings?track_id=${encodeURIComponent(trackId)}`);
const data = await res.json();
words = data.words;
const p = document.getElementById("text");
p.innerHTML = words
.map((w, i) => `<span data-i="${i}">${w.word}</span>`)
.join(" ");
}
function activeIndex(ms) {
let lo = 0, hi = words.length - 1, hit = -1;
while (lo <= hi) {
const mid = (lo + hi) >> 1;
if (words[mid].start_ms <= ms) { hit = mid; lo = mid + 1; }
else { hi = mid - 1; }
}
return (hit >= 0 && ms <= words[hit].end_ms) ? hit : -1;
}
const audio = document.getElementById("a");
audio.addEventListener("timeupdate", () => {
const ms = audio.currentTime * 1000;
const i = activeIndex(ms);
document.querySelectorAll("#text span").forEach(s =>
s.classList.toggle("lit", Number(s.dataset.i) === i));
});
load("track1");
</script>That's the entire read-along effect. The player knows nothing about books, languages, or voices — it knows words and a clock. Host it as a static page and the loop is closed: you can run HTML online and share a working reader, then publish new chapters just by loading new audio and inserting new timing rows behind the same endpoint.
Where does the friction actually live?
The parts that looked easy were easy. The parts that bit me:
Alignment drift at punctuation. The narrator pauses on a comma, and the naive end time runs long, so the highlight lingers on a word after the voice has moved on. I ended up clamping each word's `end_ms` to the next word's `start_ms` so highlights never overlap, which the SQLite editor made trivial to patch in bulk.
Tokenization has to match. The words in the table and the words rendered on screen must be the same list in the same order, or the highlight lands one word off and never recovers. I made the table the single source of truth — the player renders from the rows, not from a separately stored paragraph — so the two can't disagree.
Mobile scrubbing fires `timeupdate` sparsely. On a slow phone you can skip a word. The binary search handles it gracefully by jumping straight to whatever word the clock is inside, rather than assuming you arrived word by word.
None of this is glamorous, and none of it is a performance claim — I haven't benchmarked it at scale, so treat any speed or smoothness assertion as unverified until you test on your own content and devices. What I'll stand behind is the shape of the thing. The launch that caught my eye works because the alignment is good, and the alignment is reusable precisely because it's data, not markup. Build that half well, keep it in a table you can edit, serve it from one small endpoint, and the books and voices on top become the easy, swappable part — which is exactly backwards from how most people start.
FAQ
What is reusable in a read-along reader?
The timing map, not the book. Book text is copyrighted and varies; the mapping from position to timestamp carries across titles.
Why keep the player dumb?
So timing bugs stay in one place. A player that renders and schedules at once is impossible to debug.
What format should the timing data use?
Rows with start, duration and anchor id — easy to hand-correct and easy to diff.