Corpus Maintenance
Use for authored-source correspondence corpus maintenance: verify the vendored CIVIL/EVIL snapshot for drift against upstream, re-harvest or re-pin it, run the offline benchmark, and decide when to advance the pin.
Get the complete skill folder, including its scripts and reference files.
When to use this skill
Use Corpus Maintenance when an AI agent needs a reusable SKILL.md workflow for this job: Use for authored-source correspondence corpus maintenance: verify the vendored CIVIL/EVIL snapshot for drift against upstream, re-harvest or re-pin it, run the offline benchmark, and decide when to advance the pin.
When not to use it
Skip Corpus Maintenance when the task is outside the coding category, or when a more specific skill in this directory already covers the same workflow with clearer triggers.
How to install
Where will you use it?
mkdir -p ~/.claude/skills/corpus-maintenance # Save SKILL.md to ~/.claude/skills/corpus-maintenance/SKILL.md
- Claude Code personal install: place the skill and its bundled files at ~/.claude/skills/corpus-maintenance/SKILL.md. For a shared project, commit the folder at .claude/skills/corpus-maintenance/ instead.
- Current Claude Code detects SKILL.md changes in watched directories during the session. If you just created a top-level skills directory, run /reload-skills. Check that the skill is listed before testing its trigger.
- For Claude chat or Cowork in the Desktop app, manage uploaded skills in Customize > Skills; copying files into a local Claude Code folder is not a Desktop install. Other agents may use different install locations.
Full install guide for Claude, Cursor, and Codex →
Try a starter prompt
After installing the skill, paste this starter prompt into your agent to verify it triggers on a small task.
Use the Corpus Maintenance skill for this task: Use Corpus Maintenance when this documented workflow matches the task. Confirm any required inputs or files before making changes.
What supports this recommendation?
70/100 provisional content score. This measures documentation, practical detail and source signals. It is not a measured task success rate or a security certification.
- Documentation
- 77/100
- Practical detail
- 68/100
- Source evidence
- 60/100
- Source signals
- 72/100
Model compatibility evidence
- claudeinferred
The skill text mentions Claude or a closely associated term. This is keyword-derived compatibility; no hands-on model test is recorded.
- chatgptuntested
No model-specific signal or recorded compatibility test was found.
- geminiuntested
No model-specific signal or recorded compatibility test was found.
- copilotuntested
No model-specific signal or recorded compatibility test was found.
- llamauntested
No model-specific signal or recorded compatibility test was found.
- perplexityuntested
No model-specific signal or recorded compatibility test was found.
- mistraluntested
No model-specific signal or recorded compatibility test was found.
- grokuntested
No model-specific signal or recorded compatibility test was found.
Inferred compatibility is based on content signals. Check the app, its available tools and setup requirements before running a skill.
What this skill does
# Corpus maintenance
Use this skill when the user asks to maintain, refresh, drift-check, re-pin, or benchmark the **authored-source correspondence corpus** — the vendored source oracle behind the decompiler's offline benchmark. Produce a short status report: whether the pinned snapshot still corresponds to upstream, whether it needs a re-harvest or a pin bump, and what the benchmark says about the current decompiler.
Run from the repository root. The corpus tooling is the `DecompilerHarness` (`tools/DecompilerHarness`), a dev tool, not the shipped `dotnet-inspect` product; run it with `dotnet run --project tools/DecompilerHarness -c Release --`.
## What the corpus is
The corpus is a vendored JSONL where each row is a real method identity plus a checksum-verified authored member body captured through SourceLink at harvest time. It lets the benchmark grade source correspondence fully offline, so `SourceUnavailable` becomes a drift signal rather than an expected network outcome. It has two halves:
- **CIVIL** (Curated Index of Varied IL) — `civil/corpus.jsonl`, harvested from the 14 pinned assemblies in `eng/prepare-decompiler-corpus.sh`. This is the stable regression target that grows in lock step with the fixed real-world corpus. - **EVIL** (Edge-case Verification of IL Legibility) — `evil/corpus.jsonl`, an adversarial stress set of the most difficult real methods drawn from a much broader pool (`eng/prepare-evil-corpus.sh`), difficulty-ranked off the shared IL substrate.
Both live on the `vendor/authored-source-corpus` orphan branch so the harvested third-party source snapshots never enter `main`'s history. Restore them into a git worktree first:
```bash bash eng/restore-authored-source-corpus.sh ```
That materializes `external/authored-source-corpus`. Because it is a worktree on the orphan branch, edits made there (a re-harvest) commit directly to `vendor/authored-source-corpus`.
## The pin, and why it is stable
Every corpus assembly is a **pinned** published package version, so the only thing that varies across a decompiler change is the tool, never the input. The dotnet-inspect self-assemblies come from the published `dotnet-inspect.any` package (`SELF_VERSION` in `eng/prepare-decompiler-corpus.sh`), not the local `artifacts/bin` build — that removes corpus drift (#1404) and breaks the circularity where a decompiler change would rebuild both the tool and its own corpus at once.
Two pin axes matter:
- **Package version** — `SELF_VERSION` (and the third-party versions in `prepare-decompiler-corpus.sh`). - **Source commit** — each row's SourceLink commit, which is where its authored body was captured.
Pinning buys attribution and reproducibility; "update to latest" is a deliberate, reviewed re-pin (a re-harvest), because pins rot as upstream moves. Drift verification is the bridge that turns "should we re-pin?" into an evidence-driven decision.
## Verify drift (start here)
Drift verification re-acquires each vendored row's authored source *today* and compares it byte-for-byte (newline-normalized) against the stored snapshot. It does **not** run the decompiler; it audits the corpus, not the tool. Prepare the pinned assemblies, then verify CIVIL, resolving dotnet-inspect's own rows from local git via `--repo`:
```bash bash eng/prepare-decompiler-corpus.sh /tmp/corpus-assemblies.txt dotnet run --project tools/DecompilerHarness -c Release -- \ --verify-authored-corpus external/authored-source-corpus/civil/corpus.jsonl \ --repo "$(git rev-parse --show-toplevel)" \ $(cat /tmp/corpus-assemblies.txt) ```
Each row lands in one of three states:
- **Verified** — the re-acquired body matches the vendored snapshot. - **Drifted** — the body differs; the upstream source, the harvest slice, or the stored row itself has changed. The report names the row with a short line-count/first-diff summary. - **Unavailable** — the source could not be re-acquired or sliced (offline with no `--repo`, commit gone, checksum mismatch, or an extraction regression). The row is surfaced, never silently dropped.
It is report-only by default (exit 0). The run's own integrity still governs the exit code with a named blocker: any corpus row whose assembly was not supplied (`unmatchedRows > 0`) or a run that evaluated no rows fails, so an empty or partially-unmatched run is never a success. Add `--fail-on-drift` to make it a fail-closed gate — the run then exits nonzero unless *every* evaluated row is Verified (any Drifted or Unavailable row fails):
```bash dotnet run --project tools/DecompilerHarness -c Release -- \ --verify-authored-corpus external/authored-source-corpus/civil/corpus.jsonl \ --repo "$(git rev-parse --show-toplevel)" --fail-on-drift \ $(cat /tmp/corpus-assemblies.txt) ```
Always supply the same pinned assemblies the corpus was harvested from; a missing assembly is a blocker, not a pass.
## Interpret and decide
- **All Verified** — the pinned snapshot still corresponds to upstream. No action; the corpus is a faithful oracle. - **Drifted rows** — upstream (or the slice) moved off the pinned commit. This is expected as repositories evolve and is **not a bug** — the fix is a deliberate re-harvest / re-pin (below), not a code change. Confirm the drift is upstream churn and not a harvester regression before advancing. - **Unavailable rows** — usually an outage, a missing PDB, or a bad/incomplete `--repo` checkout, not corpus rot. Re-run with a complete `--repo` and network before concluding the source is gone. Persistent Unavailable at a known-good commit points at an extraction regression worth an issue.
Distinguish "the corpus drifted from upstream" (re-pin) from "the harness can no longer resolve known-good source" (investigate the harness).
## Re-harvest and re-pin
Advancing the snapshot is a deliberate, reviewed act. To re-harvest CIVIL from the current pins straight into the vendored worktree:
```bash bash eng/prepare-decompiler-corpus.sh /tmp/corpus-assemblies.txt dotnet run --project tools/DecompilerHarness -c Release -- \ --harvest-authored-corpus external/authored-source-corpus/civil/corpus.jsonl \ --harvest-target 12000 \ --repo "$(git rev-parse --show-toplevel)" \ $(cat /tmp/corpus-assemblies.txt) ```
`--repo` (repeatable) reads each target's authored source from a local git clone, arbitrated by the PDB checksum, falling back to the network on any mismatch or miss. Pointing it at this checkout resolves the dotnet-inspect self-corpus rows with no GitHub round-trip; it also unlocks private, large, or offline source repositories. Third-party rows are checksum-arbitrated and fall back to the network.
To **update to latest** (advance the pin, not just re-capture the same commit), bump `SELF_VERSION` in `eng/prepare-decompiler-corpus.sh` (and any third-party version), re-emit the real-world baseline, then re-harvest as above. Re-harvest EVIL from its broad pool the same way:
```bash bash eng/prepare-evil-corpus.sh /tmp/evil-pool dotnet run --project tools/DecompilerHarness -c Release -- \ --harvest-evil-corpus external/authored-source-corpus/evil/corpus.jsonl \ --harvest-target 12000 \ $(cat /tmp/evil-pool/assemblies.txt) ```
Commit the regenerated `corpus.jsonl` in the `external/authored-source-corpus` worktree (it commits to the orphan branch), and land the pin bump on `main` through a normal reviewed PR. Keep the two in step so the benchmark and the fixed real-world corpus stay aligned.
## Benchmark
The benchmark feeds the vendored authored bodies into the same source-correspondence oracle the on-demand census uses and emits a taste card. It validates the corpus and the current decompiler together; run it after a re-harvest or when asked for a decompiler correspondence read:
```bash dotnet run --project tools/DecompilerHarness -c Release -- \ --benchmark-authored-corpus external/authored-source-corpus/civil/corpus.jsonl \ $(cat /tmp/corpus-assemblies.txt) ```
Add `--json` for structured output. Each target is `Correct`, one of four valid-but-different taste buckets, `Invalid`, or a diagnostic bucket. The run exits nonzero on any `Invalid`, `Drift`, or `Unsupported`, and on a dishonest run (unmatched rows or zero targets). A `Drift` bucket here means the corpus identity no longer resolves in the pinned assembly — treat it like a Drifted verify row.
## Guardrails
- Keep the benchmark hermetic: it must stay fully offline over the vendored bodies. Do not add live network resolution to the benchmark path. - Do not delete the `vendor/authored-source-corpus` orphan branch or migrate its bodies onto `main`. The vendored snapshot is the reproducible oracle. - Build the harness with the solution graph: `dotnet build dotnet-inspect.slnx -c Release`. - Verify and benchmark over the pinned assemblies, never a fresh local build of the tool (that reintroduces #1404 circularity).
## Report shape
Keep the report short:
```md Corpus maintenance: <verified | drifted | re-pinned | benchmarked>
- Corpus: CIVIL <verified/drifted/unavailable counts>; EVIL <if checked>. - Drift cause: <none | upstream churn (re-pin) | outage/--repo (retry) | harness regression (issue)>. - Action: <none | re-harvest | bump SELF_VERSION + re-harvest | file issue>. - Benchmark (if run): <correct / taste-bucket / invalid / drift counts>, exit <code>. ```
Intended uses
- Use Corpus Maintenance when this documented workflow matches the task.
Implementation notes
- Review the source instructions and adapt inputs before running the workflow.
Try it, then tell us how it went
A copied prompt does not tell us whether the task worked. Report your result after checking the output. We collect your tool and outcome, never your files or prompt.