Skip to main content
Agent skill

Audit

Audit a Neotoma database for graph-health issues (duplicate or redundant entity types, misused types, excessive rawfragments, non-humanreadable canonical names, orphans, cycles, schema drift). Runs deterministic checks first; LLM-assisted checks are opt-in. Detection separate from repair — emits structured remediation hints pointing at existing tools (mergeentities, correct, registerschema, splitentity, deleteentity).

Get started

Get the complete skill folder, including its scripts and reference files.

When to use this skill

Use Audit when an AI agent needs a reusable SKILL.md workflow for this job: Audit a Neotoma database for graph-health issues (duplicate or redundant entity types, misused types, excessive rawfragments, non-humanreadable canonical names, orphans, cycles, schema drift). Runs deterministic checks first; LLM-assisted checks are opt-in. Detection separate from repair — emits structured remediation hints pointing at existing tools (mergeentities, correct, registerschema, splitentity, deleteentity).

When not to use it

Skip Audit when the task is outside the coding category, or when a more specific skill in this directory already covers the same workflow with clearer triggers.

How to install

Where will you use it?

Install scope
Claude Code folder setup
mkdir -p ~/.claude/skills/audit
# Save SKILL.md to ~/.claude/skills/audit/SKILL.md
  1. Claude Code personal install: place the skill and its bundled files at ~/.claude/skills/audit/SKILL.md. For a shared project, commit the folder at .claude/skills/audit/ instead.
  2. Current Claude Code detects SKILL.md changes in watched directories during the session. If you just created a top-level skills directory, run /reload-skills. Check that the skill is listed before testing its trigger.
  3. For Claude chat or Cowork in the Desktop app, manage uploaded skills in Customize > Skills; copying files into a local Claude Code folder is not a Desktop install. Other agents may use different install locations.

Full install guide for Claude, Cursor, and Codex →

Try a starter prompt

After installing the skill, paste this starter prompt into your agent to verify it triggers on a small task.

Starter prompt
Use the Audit skill for this task: Use Audit when this documented workflow matches the task. Confirm any required inputs or files before making changes.

What supports this recommendation?

69/100 provisional content score. This measures documentation, practical detail and source signals. It is not a measured task success rate or a security certification.

Documentation
77/100
Practical detail
68/100
Source evidence
56/100
Source signals
70/100
Model compatibility evidence
  • claudeinferred

    The skill uses model-agnostic prompt or LLM terminology. Portability is inferred and has not been verified on this model.

  • chatgptinferred

    The skill uses model-agnostic prompt or LLM terminology. Portability is inferred and has not been verified on this model.

  • geminiinferred

    The skill uses model-agnostic prompt or LLM terminology. Portability is inferred and has not been verified on this model.

  • copilotinferred

    The skill uses model-agnostic prompt or LLM terminology. Portability is inferred and has not been verified on this model.

  • llamainferred

    The skill uses model-agnostic prompt or LLM terminology. Portability is inferred and has not been verified on this model.

  • perplexityinferred

    The skill uses model-agnostic prompt or LLM terminology. Portability is inferred and has not been verified on this model.

  • mistralinferred

    The skill uses model-agnostic prompt or LLM terminology. Portability is inferred and has not been verified on this model.

  • grokinferred

    The skill uses model-agnostic prompt or LLM terminology. Portability is inferred and has not been verified on this model.

Inferred compatibility is based on content signals. Check the app, its available tools and setup requirements before running a skill.

What this skill does

# audit

## Purpose

Run a comprehensive read-only audit of the user's Neotoma database, surfacing graph-health issues by category and severity. Each finding includes a structured remediation pointing at an existing Neotoma tool — this skill never mutates state.

Neotoma has strong write-path discipline (idempotency, schema-first, immutability, deterministic IDs) but no holistic read-path "is my graph healthy?" tool. `/audit` fills that gap.

## Scope

**In scope:** detection of inconsistencies, drift, and quality issues across entities, observations, relationships, schemas, and raw_fragments.

**Out of scope:** mutation of any kind. Repair is performed by delegating to other tools or skills (`merge_entities`, `correct`, `split_entity`, `delete_entity`, `register_schema`, `update_schema_incremental`). The user opts into each repair individually after reviewing findings.

## Modes

- **`/audit`** — Run all deterministic checks. No LLM cost. Default. - **`/audit --deep`** — Run deterministic + LLM-assisted checks. Requires an LLM API key (read from `NEOTOMA_AUDIT_LLM_KEY` env var or passed via `--key=…`). Models default to a cheap small model; override with `--model=…`. - **`/audit --scope=<entity_type|relationship|schema|fragments>`** — Restrict checks to one category. - **`/audit --since=<ISO date>`** — Restrict to entities/observations created or updated since the date. - **`/audit --user=<user_id>`** — Audit a specific user's records (defaults to authenticated user). - **`/audit --dup-min-count=<N>`** — Minimum entity count for a type to be auto-scanned for potential duplicates beyond the always-checked `contact`/`person`/`organization` set (default 100). - **`/audit --format=<table|json|markdown>`** — Output shape; `table` default for interactive, `json` for piping into repair tooling.

## Prerequisites

- Neotoma MCP server connected (prod by default: `mcpsrv_neotoma`). - For `--deep`: LLM API key configured (`NEOTOMA_AUDIT_LLM_KEY` or `--key=…`).

## Workflow

### Phase 1 — Plan

1. Parse mode flags. 2. Resolve user scope (`get_session_identity` if no `--user` flag). 3. Print a one-line summary of what will be checked: "Running 14 deterministic checks against user X (≈N entities)". Add "+8 LLM-assisted checks" if `--deep`.

### Phase 2 — Deterministic checks

**Test-fixture filter (applies to all checks):** skip any `entity_type` matching `^(test_type_|test_activate_|cross_layer_schema_)` unless `--include-test-fixtures` is passed. These are integration-test scaffolding, not real registry entries.

Run all of the following. Each check produces zero or more `Finding` records. Each `Finding` has:

``` { check_id: "DUP_TYPE_ALIASES", severity: "high" | "medium" | "low" | "info", category: "entity_type" | "entity" | "observation" | "relationship" | "schema" | "fragments" | "naming", entity_ids: ["ent_..."], // affected entities summary: "Short human-readable summary", detail: "Longer explanation with counts/examples", remediation: { tool: "merge_entities" | "correct" | "register_schema" | …, args_hint: { ... }, // suggested args; user reviews before applying description: "Merge ent_A into ent_B as the canonical record" } } ```

#### Check list

| ID | Category | What it detects | Remediation tool | Severity | | --------------------------- | ------------- | ----------------------------------------------------------------------------------------------------------------- | --------------------------------------------- | -------- | | `DUP_ENTITIES_BY_RESOLVER` | entity | Entities flagged by `list_potential_duplicates` (server-side resolver matches). | `merge_entities` | high | | `DUP_TYPE_ALIASES` | entity_type | Two registered entity types whose `aliases` arrays overlap or whose canonical names are synonyms (e.g. `vendor` vs `merchant`). | `register_schema` (alias one to the other) | medium | | `PLURAL_TYPE_NAMES` | entity_type | Registered `entity_type` is plural and not in the irregular allowlist (per `schema_agnostic_design_rules.md`). | `register_schema` (rename to singular + alias) | medium | | `ORPHAN_ENTITIES` | entity | Entities with no `PART_OF`, `REFERS_TO`, or other inbound/outbound relationships, and no observations after the first one. | `delete_entity` or `merge_entities` | low | | `CYCLE_HIERARCHY` | relationship | Cycles in hierarchical relationship types (`PART_OF`, `DEPENDS_ON`) — explicitly forbidden by `relationships.md`. | Manual review + `delete_relationship` | high | | `UNTYPED_OR_INVALID_EDGE` | relationship | Relationships whose `relationship_type` is not in the registered enum. | `delete_relationship` + recreate with valid type | high | | `ZERO_OBSERVATION_ENTITY` | entity | Entities with snapshot but `observation_count == 0` (resolver edge case — should not happen post-bootstrap). | `delete_entity` after manual verification | medium | | `MISSING_SOURCE_PROVENANCE` | observation | Observations with null `source_id` AND null `interpretation_id` (provenance broken). | None (immutable) — investigate ingestion path | high | | `UNKNOWN_FIELDS_COUNT_HIGH` | fragments | Interpretations with `unknown_fields_count > 0` for an `entity_type` that has a registered schema (means schema is missing declared fields). | `update_schema_incremental` to add fields | medium | | `FRAGMENT_DUMP` | fragments | Entities where `raw_fragments` count exceeds the 95th percentile *and* the entity has fewer than N declared snapshot fields (signals unstructured dumping). | `register_schema` to declare common fragment_keys | medium | | `FRAGMENT_KEY_HIGH_FREQ` | fragments | `fragment_key` values whose `frequency_count` exceeds threshold T for a given `entity_type` — strong promotion candidates. | `update_schema_incremental` to promote field | info | | `NON_READABLE_CANONICAL` | naming | `canonical_name` matches a UUID pattern, hex hash, empty string, or pure punctuation/whitespace. | `correct` (provide identity-bearing field) | medium | | `MISSING_REQUIRED_FIELD` | schema | Entity snapshot missing a field declared `required` on its registered schema. | `correct` to add field | high | | `RELATIONSHIP_DANGLING` | relationship | Relationship references entity_id that does not exist or is soft-deleted (`merged_to_entity_id` set). | `delete_relationship` or update to canonical target | high | | `SCHEMA_NO_CANONICAL_FIELDS`| schema | Registered schema has empty `canonical_name_fields` — every store of that type creates a new row. | `register_schema` to declare canonical fields | high | | `INTERPRETATION_HEARTBEAT_STALE` | observation | Interpretation rows with `started_at` set, `completed_at` null, and `heartbeat_at` older than threshold. | Investigate — possibly stuck interpretation | medium |

#### Implementation notes for each check

- **`DUP_ENTITIES_BY_RESOLVER`** — Always run for the high-cardinality entity types where duplicates accumulate fastest: `contact`, `person`, `organization`. Call `list_potential_duplicates` once per type (passing `entity_type`). When `--scope=entity` or `--scope=entity_type` is not set, also include any other registered type whose entity count exceeds the per-run high-cardinality threshold (default 100; override with `--dup-min-count=N`). Skip types with fewer than 2 entities. For each returned pair, emit one `Finding`: `entity_ids` is `[entity_a.id, entity_b.id]`; `summary` names both `canonical_name` values; `detail` includes `score`, `matched_fields`, and `entity_type`; populate `args_hint` with `{ source_entity_id: entity_b.id, target_entity_id: entity_a.id, strategy: "prefer_more_recent" | "prefer_more_observations" }` (merge the lower-ranked pair member *into* the higher-ranked one — `entity_a` ranks first by score). Severity is `high` when `score >= 0.95`, otherwise `medium`. Detection only — never call `merge_entities` from a check. - **`DUP_TYPE_ALIASES`** — Call `list_entity_types`; build a graph of (canonical_name → aliases). Edges where two distinct canonical names appear in each other's alias sets, OR where alias arrays intersect, become findings. Levenshtein < 2 on short names emits as `low` severity. - **`PLURAL_TYPE_NAMES`** — `list_entity_types` and apply the same singularization rule used by `validateSchemaDefinition` (load from `NEOTOMA_ALLOWED_PLURAL_TYPES` env var or fall back to a hardcoded allowlist: `news`, `data`, `analytics`, `status`, `series`, `species`). Extend the in-skill allowlist with confirmed **legacy import artifacts** where the plural name is semantically intentional (one entity representing many items imported in bulk): `reads` (IndieWeb URL export, 2017-2018), `sections` (website section config blob), `songs`, `talking_points`, `crypto_transactions`. Emit these as `info` severity with a note that they are legacy import artifacts, not schema violations — do not co-emit `DUP_TYPE_ALIASES` for them. Validated against the live registry on 2026-05-16: surfaces `places`, `plans`, `reads`, `songs`, `sections`, `crypto_transactions`, `talking_points`, `meeting_notes` (before legacy allowlist). After applying legacy allowlist, actionable findings are `places`, `plans`, `meeting_notes` only. Three of those (`places`/`plans`/`meeting_notes`) also have singular siblings already registered — co-emit `DUP_TYPE_ALIASES` for those, ranked as the highest-priority repair candidates since rows are split across both types. - **`ORPHAN_ENTITIES`** — `retrieve_entities` per entity_type; for each entity, `list_relationships` with the entity as source or target. Empty → orphan. - **`CYCLE_HIERARCHY`** — `list_relationships` filtered by `PART_OF`/`DEPENDS_ON`; DFS cycle detection over the directed graph. - **`UNTYPED_OR_INVALID_EDGE`** — `list_relationships`; compare each `relationship_type` against the registry census from `list_relationship_types`. Do NOT hardcode a list here: the vocabulary is registrable at runtime, so a hardcoded set flags legitimately registered types as invalid. - **`ZERO_OBSERVATION_ENTITY`** — `retrieve_entity_snapshot`; check `observation_count == 0`. - **`MISSING_SOURCE_PROVENANCE`** — `list_observations` per entity (or sample); flag rows where both `source_id` and `interpretation_id` are null. Note: immutable — finding is informational unless ingestion is still actively producing them. - **`UNKNOWN_FIELDS_COUNT_HIGH`** — Aggregate `unknown_fields_count` from `interpretations` table per `entity_type`. Anything > 0 for a type with a registered schema is a finding; bucket by `fragment_key` frequency. - **`FRAGMENT_DUMP`** — Compute per-entity_type fragment counts (`raw_fragments` joined to observations). Compare each entity to the per-type 95th percentile. High count + low snapshot density = finding. - **`FRAGMENT_KEY_HIGH_FREQ`** — Group raw_fragments by `(entity_type, fragment_key)`, sum `frequency_count`. Threshold defaults: keys appearing on >25% of entities of that type. - **`NON_READABLE_CANONICAL`** — Regex against `canonical_name`: UUID (`^[0-9a-f]{8}-…`), hex (`^[0-9a-f]{16,}$`), empty, whitespace-only, single-character. - **`MISSING_REQUIRED_FIELD`** — Load schema definitions via `list_entity_types`; for each entity, diff snapshot keys against `required_fields`. - **`RELATIONSHIP_DANGLING`** — `list_relationships`; for each row, verify `source_entity_id` and `target_entity_id` resolve via `retrieve_entity_snapshot` and are not soft-deleted. - **`SCHEMA_NO_CANONICAL_FIELDS`** — `list_entity_types`; flag any with empty `canonical_name_fields`. - **`INTERPRETATION_HEARTBEAT_STALE`** — Query interpretation rows; threshold defaults to 15 minutes.

### Phase 3 — LLM-assisted checks (only when `--deep`)

Run these only if the API key is present. Each check batches entities and asks the LLM for structured JSON output. Key never leaves the local process.

| ID | What it detects | Remediation tool | | ----------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------- | | `LLM_CANONICAL_READABILITY` | `canonical_name` is technically valid (passes deterministic regex) but not human-readable (e.g., `payment-3-2`, `unnamed-vendor-7`). | `correct` with suggested name | | `LLM_ENTITY_TYPE_MISCLASS` | Content semantically a `transaction` but stored as `note`/`message`/`raw`. LLM compares snapshot+observations to type registry. | `correct` with `entity_type` change | | `LLM_SEMANTICALLY_EQUIVALENT_TYPES` | Two entity_types not alias-linked but semantically equivalent (`vendor` vs `merchant` vs `payee`). | `register_schema` (alias) | | `LLM_FRAGMENT_PROMOTION` | Raw fragments that are stable, structured, and high-value enough to promote to declared schema fields. | `update_schema_incremental` with suggested field shape | | `LLM_RELATIONSHIP_SUGGESTION` | Pairs of entities whose observation text strongly implies a relationship that isn't declared. | `create_relationship` with suggested type | | `LLM_DUPLICATE_BY_CONTENT` | Entities that escaped resolver-based duplicate detection but have semantically equivalent content (different spellings, abbreviations). | `merge_entities` |

### Phase 4 — Report

Output sections, in this order:

1. **Summary line** — counts by severity: `Found 47 issues (3 high, 12 medium, 32 low/info)`. 2. **Potential duplicates** — a dedicated section for all `DUP_ENTITIES_BY_RESOLVER` findings, broken out from the generic severity lists because duplicate accumulation is the most common silent graph-health failure. Group by `entity_type`; within each type list the candidate pairs ranked by `score` (truncate to 10 with `(+N more)` unless `--verbose`). Each row shows both `canonical_name` values, the `score`, the `matched_fields`, and the merge hint. End the section with the always-checked types that returned zero candidates so the operator sees the check ran (`contact: no candidates`). If no pairs were found across any type, print `Potential duplicates: none detected`. Remediation is `merge_entities` only — emit the hint, never merge. 3. **High-severity findings** — list each remaining (non-duplicate) high finding with `check_id`, summary, affected entity ids (truncated to 5 with `(+N more)`), and remediation one-liner. 4. **Medium-severity findings** — same format, collapsed by default in interactive mode. 5. **Low/info findings** — collapsed group; show count only unless `--verbose`. 6. **Repair menu** — numbered list of remediation actions the user can authorize: `1. Merge 7 duplicate pairs (merge_entities)`, `2. Rename 2 plural entity types (register_schema)`, etc. The user picks individually; no batch auto-apply.

### Phase 5 — Repair (delegated, opt-in)

For each remediation the user authorizes:

1. Look up the target tool from the finding's `remediation.tool`. 2. Print the exact tool call that will be made, with `args_hint` filled in. 3. Wait for explicit confirmation per finding (or `--yes` to apply all of one category). 4. Invoke the tool. Record the result alongside the finding. 5. After repair, re-run the affected check class to verify the issue is resolved.

**Never auto-apply.** Repair is always per-finding (or per-category with `--yes`), never per-audit-run.

## Constraints

- MUST NOT mutate any Neotoma state in detection phases. - MUST emit a `Finding` for every issue surfaced; no silent skips. - MUST include a `remediation.tool` pointing at an existing Neotoma tool (or `null` with rationale if no automated repair is possible). - MUST NOT call LLM checks unless `--deep` and a key is present. - MUST batch LLM calls and cap total LLM cost; print an estimate before running deep mode and require confirmation if estimate exceeds `$NEOTOMA_AUDIT_COST_CAP` (default $1). - MUST respect `--user` scope and refuse to audit other users' data without explicit authorization (`get_session_identity`). - MUST run deterministic checks before LLM checks; LLM checks may consume deterministic findings as input but never vice versa. - MUST store the audit run itself as a Neotoma entity (`entity_type: audit_run`, with `findings_count`, `started_at`, `completed_at`, `mode`, `scope`) for historical comparison.

## Forbidden patterns

- Calling `merge_entities`, `correct`, `delete_entity`, etc., without explicit user confirmation per finding. - Inferring entity types or relationships from deterministic checks (those are LLM-mode only). - Failing silently on tool errors — surface every error as an `info` finding with the tool response. - Storing LLM API keys in Neotoma or logs. - Generating findings from heuristics not declared in this skill (no ad-hoc checks).

## Output examples

### Default `/audit` (deterministic only)

``` Auditing Neotoma database for user mark (≈8,420 entities) Running 14 deterministic checks…

Summary: 47 findings (3 high, 12 medium, 32 low/info)

Potential duplicates (DUP_ENTITIES_BY_RESOLVER) — 7 candidate pairs contact (5 pairs) "Jane Doe" ↔ "Jane R. Doe" score 0.97 [canonical_name, email] → merge_entities(source=ent_bbb…, target=ent_aaa…, strategy=prefer_more_observations) "Acme, Inc." ↔ "Acme Inc" score 0.93 [canonical_name] → merge_entities(source=ent_ddd…, target=ent_ccc…, strategy=prefer_more_recent) … (+3 more, run --verbose) organization (2 pairs) "Vercel" ↔ "Vercel Inc." score 0.91 [canonical_name] → merge_entities(source=ent_fff…, target=ent_eee…, strategy=prefer_more_recent) person: no candidates

HIGH (2) [CYCLE_HIERARCHY] 1 cycle detected in PART_OF graph → ent_ccc → ent_ddd → ent_eee → ent_ccc → Remediation: manual review + delete_relationship

[SCHEMA_NO_CANONICAL_FIELDS] entity_type "note" has no canonical_name_fields → Every store creates a new row; deduplication impossible → Remediation: register_schema with canonical fields

MEDIUM (12) — collapsed, run with --verbose

LOW/INFO (32) — collapsed, run with --verbose

Repair menu: 1. Merge 7 potential duplicate pairs (merge_entities, review each) [y/n] 2. Resolve cycle in PART_OF graph (manual) [skip] 3. Add canonical_name_fields to "note" schema [y/n] Run --verbose for medium/low remediations. ```

### `/audit --deep --scope=naming`

``` Auditing naming category only (deep mode, model gpt-4o-mini) Estimated LLM cost: $0.18 — proceed? [y]

Running 1 deterministic check (NON_READABLE_CANONICAL)… Running 1 LLM check (LLM_CANONICAL_READABILITY)…

Summary: 23 findings (0 high, 18 medium, 5 info)

MEDIUM (18) [NON_READABLE_CANONICAL] 12 entities with UUID-shaped canonical_name [LLM_CANONICAL_READABILITY] 6 entities with technically-valid but unreadable names → ent_fff "payment-3-2" → suggested "Acme Corp - Invoice #3 - Feb 2026" → ent_ggg "vendor-unnamed-7" → suggested "Cloudflare" → … (4 more) → Remediation: correct (with LLM-suggested names)

Repair menu: 1. Apply 12 deterministic name corrections (correct, manual) [y/n] 2. Apply 6 LLM-suggested name corrections (correct, review each) [y/n] ```

## Related

- `docs/foundation/schema_agnostic_design_rules.md` — singular type names, schema-driven behavior. - `docs/subsystems/observation_architecture.md` — raw_fragments lifecycle. - `docs/subsystems/relationships.md` — typed edges, cycle prohibition. - `docs/subsystems/schema.md` — `raw_fragments`, `interpretations` schema. - `docs/subsystems/entity_merge.md` — merge semantics. - `docs/foundation/entity_resolution.md` — resolver path and identity rules.

## Promotion path

This skill is the prototype shape. Once the check set stabilizes (≥2 weeks of use, deterministic checks ratified) promote to a first-class CLI command:

- `neotoma audit [--deep] [--scope=…] [--since=…] [--format=…]` - OpenAPI contract update per `docs/architecture/openapi_contract_flow.md`. - MCP tool exposure for cross-harness use. - Feature unit under `docs/feature_units/in_progress/FU-…audit/`.

Intended uses

  • Use Audit when this documented workflow matches the task.

Implementation notes

  • Review the source instructions and adapt inputs before running the workflow.

Try it, then tell us how it went

A copied prompt does not tell us whether the task worked. Report your result after checking the output. We collect your tool and outcome, never your files or prompt.