There's a V2EX post that'll feel familiar to anyone who's tried to ship a quiz app: someone asking where to *buy* a database. Not any database — specifically one scraped from the main question-bank sites and packaged for resale. Their reasoning was honest and relatable. Pulling questions off those sites by hand was brutal, one page at a time. And the official route, hooking into an API, was closed to them because individual developers don't get access; you need company credentials.
So the instinct was to shortcut the whole thing by buying a dump someone else scraped. I get the appeal. But I think it's solving the wrong problem, and the shortcut quietly becomes the thing that sinks the project later.
The bought dump is a liability, not an asset
Here's the uncomfortable part. A scraped dump is someone else's extraction, with someone else's mistakes baked in. Duplicate questions phrased three slightly different ways. Options that got mangled in whatever parser produced the file. Answers keyed to a format you'll have to reverse-engineer. You pay money, and you still spend a week cleaning it before a single question renders correctly in your app.
Worse, it's not really *yours*. You can't trace where it came from, you can't vouch for its accuracy, and you can't legally stand behind redistributing content that was lifted from sites that deliberately refused you API access. The forum poster already hit the wall those sites built on purpose. Buying a dump doesn't knock the wall down — it just moves the liability onto you and hides it.
I want to be straight about the boundary here, because it matters. VicroCode isn't a scraper, and it won't get you into a third-party site's API that only hands out company credentials. If those are the doors you're trying to force, nothing I'm about to describe opens them. That's the hard edge of this problem, and no tool erases it.
Reframe: the durable asset is a curated item store
So change the question. Instead of "where do I buy the data," ask "what's the asset I actually want to own?
The answer is a clean, deduplicated item store you built and understand — one you can grow from legitimate inputs over time: questions you write, material you're licensed to use, content colleagues hand you as CSV or pasted text or JSON exports from tools you already pay for.
The moment you own the *store* rather than renting a *dump*, the manual-download pain stops being a dead end and becomes a one-time import you only run when you've got something clean to add. Three pieces make that real, and all three map directly onto what you can stand up in one place.
1. A SQLite schema that treats questions as first-class data
Start with the shape of a question. You don't need anything exotic. A `questions` table with the stem and a stable hash of the normalized text. An `options` table linked back by question id, with the option text and an order index. An `answers` table marking which option (or options) is correct. Maybe a `source` column so you always know where each item entered your store, and a `tags` relationship so you can slice quizzes by topic later.
The hash column is the quiet hero. If you normalize each question's text — trim whitespace, collapse casing differences, strip trailing punctuation — and hash that normalized form, duplicate detection becomes a unique index instead of a guessing game. You can model all of this directly with a SQLite editor, building and inspecting the tables as you go, so the schema is something you can see and adjust rather than a black box inside your app.
2. A Python importer that normalizes and dedups whatever you feed it
This is where the "manual downloads are brutal" complaint actually gets solved. Write one importer that accepts the messy reality of your inputs — a CSV from a spreadsheet, a block of pasted text in a loose format, a JSON array from some export. The importer's job is to coerce all of it into the same internal shape: a stem, a list of options, a flagged correct answer, an optional set of tags.
Then it normalizes each stem the same way your schema expects, computes the hash, and checks whether that question already lives in the store. New item? Insert it. Already there?
Skip it, or merge the tags, your call. Because the dedup logic lives in code you wrote, you can tune it. Maybe you decide two questions with identical stems but different option orderings are the same item. Maybe you don't. That's a product decision, and it stays in your hands.
You can run Python online for exactly this kind of job — point the importer at a file or a pasted blob, watch what it inserts and what it rejects, and iterate on the normalization rules until the store stays clean. Every time you feed it a new batch, it's a small, repeatable operation instead of the one-page-at-a-time grind that pushed the original poster toward buying a dump.
3. A hosted HTML quiz front end over the store
The last piece is the part users touch. A plain HTML quiz app that reads from your item store, shows a question with its options, records the answer, and moves on. Nothing heavy. The whole point of doing the hard curation work upfront is that the front end gets to be simple, because the data behind it is already clean.
Since you can run HTML online and publish it from the same place the data lives, you don't stitch together a separate host, a separate database, and a separate deploy step. The quiz, the SQLite store, and the Python importer all sit together, which is what makes the "grow it over time" story practical instead of aspirational.
What this actually buys you
Think about where the two paths lead. The bought-dump path: you spend money once, inherit someone else's duplicates and errors, can't vouch for or safely redistribute it, and you're back to square one the day you want fresh content. The owned-store path: you spend effort building an importer once, and after that every clean input you find — authored, licensed, contributed — slots in with automatic dedup and a schema you control.
I won't pretend the second path is free. Writing the normalizer takes real thought, and the first import always surfaces edge cases you didn't anticipate. But the trade is the right one. You're converting a one-time purchase of a decaying asset into a system that compounds. That's the difference between renting data and owning an item store.
And it keeps you on the right side of the line that blocked the original poster in the first place. You're not trying to pry open an API meant for companies or redistribute someone's scrape. You're building a store from inputs you can actually stand behind, with the dedup and curation logic sitting in code you can read. The brutal manual downloads were a symptom. The real fix was never a better dump to buy — it was owning the thing the dump was pretending to be.