Streaming LLM Output Into a Live Table Without the Half-Parsed Garbage

I kept coming back to one line buried in a developer's job post. Eight years of experience, full-stack, the usual stack, and then this almost throwaway detail in the AI section: they'd built "streaming-generated content table rendering." One phrase. No elaboration. But anyone who has tried to paint rows into a table while a model is still talking knows exactly how much pain is folded into those few words.

So I rebuilt the winnable half of it, and I want to walk through where it holds up and where it quietly falls apart.

The problem nobody puts on a slide

When a model streams output, you get tokens, not records. The model doesn't hand you row one, then row two. It hands you a trickle of characters, and somewhere inside that trickle are the boundaries of your data. If you try to parse what you have on every chunk, you spend most of the stream looking at fragments: half a cell, an open brace with no close, a quoted string that hasn't been terminated. Render that eagerly and your table flickers with garbage, columns jump around, and a user watching it feels like the screen is glitching.

The naive fix is to wait for the whole response and parse once. That works and it's boring, and it throws away the entire reason you wanted streaming: the sense that something is happening. The interesting version keeps the live feel but refuses to render anything that isn't finished.

The idea that makes it tractable

The trick is to pick an output format where each unit is self-contained and separated by an unambiguous delimiter. Newline-delimited JSON is the friendliest version I've used. You ask the model for one JSON object per line, one row per object. Then the backend only needs one rule: never emit a row until you've seen the newline that closes it. Everything before that newline is a buffer, not a result.

That single rule is what kills the half-parsed garbage. You're not trying to reconstruct arbitrary structure mid-flight. You're waiting for a boundary you can trust, parsing the complete chunk, and only then telling the frontend "here's a committed row."

Building it on confirmed pieces

I put the model-facing logic in a Python backend, because the buffering and parsing belong on the server where you control the loop. VicroCode lets you run Python online and host it as an API endpoint, so there's no separate server to stand up. The endpoint calls a model that's already available through Model Center, reads the stream, and holds characters in a buffer until a newline arrives.

The shape of the server loop is small:

import json

def stream_rows(model_stream):
    buffer = ""
    for token in model_stream:          # tokens from a Model Center model
        buffer += token
        while "\n" in buffer:
            line, buffer = buffer.split("\n", 1)
            line = line.strip()
            if not line:
                continue
            try:
                row = json.loads(line)   # only commit a fully-formed object
            except json.JSONDecodeError:
                continue                 # not a clean row yet, keep waiting
            yield json.dumps(row) + "\n" # forward only committed rows

The `try/except` is doing honest work here. If a line arrives malformed, you skip it instead of guessing. That's a deliberate choice: a dropped row is recoverable, a corrupted table is not.

The frontend is just as plain. VicroCode can host and run HTML online, so a single page reads the response stream, splits on newlines, and appends one `<tr>` per committed row. No framework required.

<table id="out"><tbody></tbody></table>
<script>
async function render(url) {
  const res = await fetch(url);
  const reader = res.body.getReader();
  const dec = new TextDecoder();
  let buf = "";
  const body = document.querySelector("#out tbody");
  while (true) {
    const { value, done } = await reader.read();
    if (done) break;
    buf += dec.decode(value, { stream: true });
    let i;
    while ((i = buf.indexOf("\n")) >= 0) {
      const line = buf.slice(0, i).trim();
      buf = buf.slice(i + 1);
      if (!line) continue;
      const row = JSON.parse(line);
      const tr = document.createElement("tr");
      tr.innerHTML = Object.values(row)
        .map(v => `<td>${String(v)}</td>`).join("");
      body.appendChild(tr);
    }
  }
}
</script>

Notice the frontend buffers too. The network can split a chunk anywhere, including mid-line, so the client applies the same "wait for the newline" discipline as the server. Both ends agree that a row isn't real until its delimiter shows up.

One security note worth saying out loud: this endpoint as written is open. If you host it, put some access control in front before you point it at a billed model, or you're paying for anyone who finds the URL.

Where it breaks, honestly

The line-delimited approach wins because the structure is flat and the delimiter is reliable. The moment you want something richer, the ground gets soft.

If the model emits a single large JSON array instead of one object per line, you're back to parsing an incomplete structure, and there's no safe boundary until the closing bracket. If rows contain nested objects whose own braces look like boundaries, a dumb delimiter split will cut them in the wrong place. If the model decides to reorder or revise an earlier row partway through, your append-only table can't represent that without rework. And models don't always honor the format. Sometimes you get prose before the first row, a trailing apology after the last one, or a line that's valid JSON but the wrong shape.

That's the part the resume line glosses over. "Streaming table rendering" sounds like one feature. In practice it's a clean, shippable core wrapped in a set of edge cases that each need a decision. I handled the common ones by skipping unparseable lines and ignoring non-object lines, but "skip it" is a product choice, not a universal answer. For a financial or reporting table where a missing row is a real problem, you'd want the backend to collect skipped lines and surface them, not swallow them.

Why this is a good use of an afternoon

The reason I bothered reconstructing this is that it's a genuinely reusable pattern, not a one-off. Any time you're turning model output into something tabular that a human watches fill in, the server-buffers-until-boundary, client-renders-committed-rows split applies. Leaning on a tool to scaffold the parsing loop and the fetch reader is exactly the kind of task where AI coding earns its keep, because the logic is fiddly but well-understood, and the mistakes are easy to spot once something renders.

The honest summary: the live-table feel is real and shippable with flat, delimited rows and a strict commit rule on both ends. The messy, nested, revisable version of structured streaming is where you should expect to spend your real debugging time, and no amount of clever parsing makes an incomplete structure safe to render before it's complete.