A builder recently shared the internals of Parallax, a production arXiv search system, and buried in the write-up is the one design decision I think travels best to other projects. They started with the obvious funnel: recall, coarse filter, fine filter. Then they noticed paper search doesn't behave like a recommendation feed. You need to expand a result set by chasing citations, and you need to go back and review what the first pass missed or wrongly let in. A one-way funnel can't do either. So they rewrote the whole retrieval core around set algebra, with paper sets as the basic unit and operators like union and difference on top.
That reframing is the part I'd steal. The semantic recall quality and the model-driven fine-pass get all the glory, but they're also the parts most tied to whatever model you happen to be using this month. The set operations underneath are plain, deterministic bookkeeping, and that's exactly why they last. Handing large-scale merge and subtract work to a deterministic program also protects you against model hallucination and context-window amnesia, which is the quiet reason the approach holds up at scale.
What the set layer actually does
Strip away the branding and the core is small. Multi-path recall throws keyword hits and semantic hits into a pool. Union merges those paths and dedups by a stable key. Difference subtracts a set of false positives you've flagged. A virtual root lets you name a starting set, like the whole library or a day's announcements, without persisting millions of rows. Rule filters narrow by category, date window, or similarity threshold. The model-driven steps, refine and classify, sit on top of this ledger rather than replacing it.
Parallax's own cases show the shape. One run on an agent-memory topic fired six keyword paths and five semantic paths, unioned them into a thirty-thousand-plus candidate set, then sent that to the fine-pass. A tighter multimodal run pulled 466 candidates down three paths, refined to 428 matches, then classified into ten buckets that later split into eleven sub-buckets. In both cases the interesting agent behavior is pure set manipulation: build a set, merge it, prune it, slice it. The model only judges membership.
Rebuilding the core on VicroCode
Here's how I'd reconstruct that durable half with the capabilities actually on hand, and where the walls are.
The candidate ledger goes in SQLite. One table for papers keyed by arXiv id, one for set membership, one to record which recall path and run produced each row, plus a dedup key. Every set operation becomes a query you can read and audit later. Being able to open the tables directly through the SQLite editor matters more than it sounds, because when an agent builds a tangled workflow you want to inspect the intermediate sets by hand instead of trusting a log line.
The recall layer goes into a LanceDB knowledge base holding the abstract embeddings. That's the vector store the semantic paths query against. Worth being honest about the boundary immediately: Parallax ran on PostgreSQL with pgvector and reported roughly 17GB of vectors and indexes for about 820,000 papers. That specific stack isn't what you'd use here. You model the recall layer with LanceDB instead, and you don't get to assume pgvector's exact indexing behavior carries over.
The set operations themselves become a hosted Python endpoint. Union, dedup, and subtract are a few dozen lines over the SQLite ledger, and you can run Python online and expose them through API Endpoint Hosting so an agent calls them as in-platform tools. This is the piece I'd trust most, because it's deterministic and testable. Given the same inputs, union(A, B) returns the same set every time, which is the whole point of pushing this work out of the model's head and into code.
Where the honest boundary sits
Two things ride on the model, and no amount of clean set algebra fixes that. First, semantic recall quality depends entirely on the embedding model. Parallax used a specific embedding model to turn abstracts into 1024-dimension vectors; on VicroCode you're limited to whatever's reachable through the Model Center APIs, so your match quality is bounded by the models already available on the platform, not by the exact one in the source write-up. Second, the fine-pass judgment is a model call. The source system leaned on a type-safe batch primitive called Jev for refine and classify, and reported that switching to it refined 36,564 papers in about five minutes, over ten times faster than their LLM pipeline at under a fifth of the cost. Treat those figures as unverified. They're the builder's own reported results on their own stack, not something reproducible on VicroCode, and Jev itself isn't a capability here. Your refine step would be a Model Center call, with latency and cost set by whatever model you pick.
So the split is clean. The set layer, the ledger, and the recall plumbing are things you can actually build and verify. The semantic quality and the fine-pass economics stay contingent on the model, and you should say so out loud rather than promising numbers you can't stand behind.
Why this is worth doing
The reason to care goes beyond one arXiv tool. Across a lot of what independent builders ship, the fragile part is the model-dependent glamour and the durable part is the boring data structure underneath. A set-algebra ledger you can union, dedup, and subtract gives you reproducibility, auditability, and a core that survives swapping models. Build that on a SQLite ledger, a LanceDB recall layer, and a hosted Python endpoint, keep the model behind a clear boundary, and you've got the half of the system that keeps working after the model hype cycle moves on. Then you can publish and share the whole thing from the same platform instead of stitching together hosting elsewhere.