A thread that stuck with me this week was a plain complaint: you keep installing skill packages for your AI, and after a while it turns into a knot. Same skill saved three times, some of them long dead, and no clear answer to which one is actually in effect. The person behind it shipped a tool-management view for a sessions viewer to make the duplicates and broken ones visible at a glance, then delete them cleanly. That's the whole problem in one sentence — and it's exactly the kind of thing that quietly rots until it bites you mid-task.
I've lived the smaller version of this. You copy a skill folder to try something, tweak it, forget the original, then months later two definitions with the same name sit in different directories and you have no idea which one the harness picks up. The instinct is to reach for a fancy runtime debugger. But most of the pain isn't runtime — it's inventory. You simply don't have a clean list of what's installed, where it lives, and whether it's still valid. That's a durable, boring problem, and boring problems are the ones worth automating.
So here's the build I'd reconstruct, and where I'd honestly draw the line.
The durable half: catalog first, judge second
The part that survives across tools and updates is the catalog. Not "which skill won the runtime coin flip," but "what definitions exist on disk and what's their state." That's a scanner, and a scanner is a great fit for Python. You walk the skill directories, read each definition file, and record four things per entry: the path, a content hash, a name or identifier, and a status.
Status is where the value lives. I'd keep it to a short, honest vocabulary:
- **active** — parses cleanly, has the fields a skill needs, no obvious problem.
- **duplicate** — same identifier or identical hash as another entry, just in a different location.
- **broken** — won't parse, missing required fields, or points at something that isn't there.
- **shadowed** — a valid definition that is almost certainly overridden by another entry with the same name, based on load-order rules you can observe on disk.
The hash is what makes duplicate detection cheap and reliable. Two files with the same hash are the same skill, full stop. Two files with the same name but different hashes are the interesting case — that's where you've been editing copies and losing track. You can run Python online to do the whole walk-and-classify pass without setting up a local environment, which matters if you're auditing skills that live in a project you're already working on inside the platform.
One thing I'd resist: don't try to be clever in the scanner. Classify conservatively. If you can't confidently call something broken, call it active and let the human look. A scanner that cries wolf gets ignored within a day.
Store it somewhere you can actually correct
The scanner's output isn't a report you print once — it's a working table you'll revisit. That's the case for putting every entry into SQLite rather than dumping JSON. A row per skill definition, columns for path, hash, name, status, and a note field. When the scanner is wrong, or you've made a decision ("yes, this duplicate is intentional, leave it"), you want to fix the record without rerunning everything.
That's where an in-browser SQLite editor earns its place. You re-scan, the table updates, and when a classification looks off you edit the row directly and move on. Keeping the note field human-owned is deliberate: the scanner writes status, you write the reasoning. Next time you run it, you can diff against the stored state and see what changed — new duplicates that appeared, entries that flipped from active to broken after an edit. That diff is honestly more useful than any single snapshot.
A small trade-off I'd flag: if you re-hash and re-key on path every run, moved files look like a delete plus an add. For a first version that's fine — you're auditing, not doing version control. If it annoys you, key on the identifier instead of the path and treat path as a mutable column.
Surface the conflicts where you'll see them
A table is for correcting; a report is for noticing. The last piece is a read view that makes conflicts obvious without opening a database. I'd generate a plain HTML page grouped by problem: duplicates clustered by name, broken entries with their parse error, shadowed skills next to the entry that shadows them. Green for active, and everything else pulled to the top so the noise is impossible to miss.
Standing that up through web app hosting gives you a link you can open anytime or share with a teammate who maintains the same skill set. No install, no "works on my machine." The report reads from the same SQLite data, so your manual corrections show through — a duplicate you marked intentional stops screaming at you.
If you want it to breathe on its own, you could wrap the scan behind a small hosted endpoint so re-running the audit and refreshing the report is one action. That's a nice-to-have, not the core. The core is: catalog, correct, surface.
Where I'd stop, and say so plainly
Here's the honest boundary, because pretending otherwise would make the tool worse than useless. This build tells you what's installed and what conflicts exist. It does **not** tell you, with authority, which skill your AI harness actually loads at runtime. That decision belongs to the harness — its own resolution order, its caching, its config — and observing files on disk is an inference, not a guarantee. My "shadowed" status is a well-reasoned guess based on load rules you can see, not a verified runtime trace.
So treat the report as a map of the mess, not a live probe of the running system. It's genuinely good at the thing that rots quietly — three copies of one skill, a definition that's been broken for a month, a name collision you forgot about. It will not sit inside your agent's execution and confirm the winner. If someone tells you a purely file-based scanner can prove runtime behavior, that claim is unverified, and I'd push back on it.
The reason I'd still build it: the inventory problem is where almost all the actual pain comes from, and it's the part you can own completely. Get the catalog clean and editable, put the conflicts on a page you'll actually look at, and the runtime question shrinks to "which of these two I already know about wins" — a much smaller thing to reason about than a knot you can't see.