skillcheck update fixes scoring, adds clean failures and measured token error rates

What shipped

skillcheck, a static analyzer for SKILL.md files, received an update focused on hardening and accuracy rather than new features. SKILL.md is the format that agents including Claude Code, Copilot, Codex and Cursor use to load reusable skills; the tool validates frontmatter, scores how discoverable a skill's description is, checks file references, enforces tiered token budgets and flags cross-agent compatibility problems. According to the maintainer's post on dev.to, it performs no network calls, makes no LLM API calls and never mutates files, and runs as a CLI, a GitHub Action or a pre-commit hook. It targets the agentskills.io specification and requires Python 3.10 or later.

A rebuilt description scorer

The most visible change is in the description scorer. The previous implementation was under-crediting well-written descriptions, which matters because the score exists to predict whether an agent will actually find and trigger a skill. After the fix, the median score across the tool's reference corpus rose from 75 to 90. The --explain-score flag now reports which scoring pattern hit or missed instead of returning a bare number, and the maintainer says the fix was validated against real-world skills: filler text still scores between 28 and 65, while well-written descriptions land between 85 and 100, so the separation between good and bad copy survives.

Corrupt files fail cleanly

The update also replaces crashes with clean errors. Previously, a corrupt history ledger or a non-UTF-8 skillcheck.toml located above a skill directory produced a raw Python traceback. Because configuration discovery walks up the directory tree, a single malformed file could break every scan beneath it. Now all untrusted reads — file ingestion, history and configuration — pass through a common guard before parsing, so each one fails the same way: a clear error naming the file and the byte offset, with exit code 2.

Measured token estimates

The README's token numbers have been corrected to reflect measurement rather than assumption. The offline heuristic that feeds skillcheck's budget checks had never actually been benchmarked; it now has been, against tiktoken's cl100k_base tokenizer across a corpus of 61 real SKILL.md files. The bias is one-directional: the heuristic over-estimated on all 61 files. Median error is 23.0% for whole-file sizing (p95 30.7%), 25.9% for frontmatter metadata budgets (p95 35.3%), and 22.7% for body budgets (p95 30.7%). Without the optional tiktoken extra, users should expect roughly 20-30% over-estimation, so the maintainer recommends installing the skillcheck[tiktoken] extra when working near a budget limit.

Internal hardening

The remainder of the release is deliberately invisible: flag-conflict logic was consolidated into a single source of truth, golden-file tests now pin exact diagnostic output, and the coverage floor was raised from 75% to 80%, with actual coverage sitting at 90% across 1,058 tests in the rule modules. Diagnostic output across the corpus was verified byte-for-byte identical before and after the change, meaning nothing outside the items above shifted.

Why it matters

SKILL.md is becoming shared infrastructure across several major coding agents, and quality gates for it are only useful if their signals are trustworthy. A scorer that punishes good descriptions, or a tokenizer that silently overcounts by a quarter, can push authors toward the wrong edits — trimming text that was fine or gaming a metric that was broken. By measuring its own error rates, failing loudly on corrupt input and keeping diagnostics stable, skillcheck is holding itself to the same standard it applies to the skills it audits.