Skip to main content
Agent skill

Agent Architecture Audit

Full-stack diagnostic for agent and LLM applications. Audits the 12-layer agent stack for wrapper regression, memory pollution, tool discipline failures, hidden repair loops, and rendering corruption. Produces severity-ranked findings with code-first fixes. Essential for developers building agent applications, autonomous loops, or any LLM-powered feature. Use when an agent or LLM feature misbehaves and the failing layer is unknown, or before shipping an agent stack.

Get started

Get the complete skill folder, including its scripts and reference files.

·

When to use this skill

Use Agent Architecture Audit when an AI agent needs a reusable SKILL.md workflow for this job: Full-stack diagnostic for agent and LLM applications. Audits the 12-layer agent stack for wrapper regression, memory pollution, tool discipline failures, hidden repair loops, and rendering corruption. Produces severity-ranked findings with code-first fixes. Essential for developers building agent applications, autonomous loops, or any LLM-powered feature. Use when an agent or LLM feature misbehaves and the failing layer is unknown, or before shipping an agent stack.

When not to use it

Skip Agent Architecture Audit when the task is outside the coding category, or when a more specific skill in this directory already covers the same workflow with clearer triggers.

How to install

Where will you use it?

Install scope
Claude Code folder setup
mkdir -p ~/.claude/skills/agent-architecture-audit
# Save SKILL.md to ~/.claude/skills/agent-architecture-audit/SKILL.md
  1. Claude Code personal install: place the skill and its bundled files at ~/.claude/skills/agent-architecture-audit/SKILL.md. For a shared project, commit the folder at .claude/skills/agent-architecture-audit/ instead.
  2. Current Claude Code detects SKILL.md changes in watched directories during the session. If you just created a top-level skills directory, run /reload-skills. Check that the skill is listed before testing its trigger.
  3. For Claude chat or Cowork in the Desktop app, manage uploaded skills in Customize > Skills; copying files into a local Claude Code folder is not a Desktop install. Other agents may use different install locations.

Full install guide for Claude, Cursor, and Codex →

Try a starter prompt

After installing the skill, paste this starter prompt into your agent to verify it triggers on a small task.

Starter prompt
Use the Agent Architecture Audit skill for this task: Use Agent Architecture Audit when this documented workflow matches the task. Confirm any required inputs or files before making changes.

What supports this recommendation?

71/100 provisional content score. This measures documentation, practical detail and source signals. It is not a measured task success rate or a security certification.

Documentation
77/100
Practical detail
68/100
Source evidence
63/100
Source signals
72/100
Model compatibility evidence
  • claudeinferred

    The skill uses model-agnostic prompt or LLM terminology. Portability is inferred and has not been verified on this model.

  • chatgptinferred

    The skill uses model-agnostic prompt or LLM terminology. Portability is inferred and has not been verified on this model.

  • geminiinferred

    The skill uses model-agnostic prompt or LLM terminology. Portability is inferred and has not been verified on this model.

  • copilotinferred

    The skill uses model-agnostic prompt or LLM terminology. Portability is inferred and has not been verified on this model.

  • llamainferred

    The skill uses model-agnostic prompt or LLM terminology. Portability is inferred and has not been verified on this model.

  • perplexityinferred

    The skill uses model-agnostic prompt or LLM terminology. Portability is inferred and has not been verified on this model.

  • mistralinferred

    The skill uses model-agnostic prompt or LLM terminology. Portability is inferred and has not been verified on this model.

  • grokinferred

    The skill uses model-agnostic prompt or LLM terminology. Portability is inferred and has not been verified on this model.

Inferred compatibility is based on content signals. Check the app, its available tools and setup requirements before running a skill.

What this skill does

# Agent Architecture Audit

A diagnostic workflow for agent systems that hide failures behind wrapper layers, stale memory, retry loops, or transport/rendering mutations.

## When to Activate

**MANDATORY for:** - Releasing any agent or LLM-powered application to production - Shipping features with tool calling, memory, or multi-step workflows - Agent behavior degrades after adding wrapper layers - User reports "the agent is getting worse" or "tools are flaky" - Same model works in playground but breaks inside your wrapper - Debugging agent behavior for more than 15 minutes without finding root cause

**Especially critical when:** - You've added new prompt layers, tool definitions, or memory systems - Different agents in your system behave inconsistently - The model was fine yesterday but is hallucinating today - You suspect hidden repair/retry loops silently mutating responses

**Do not use for:** - General code debugging — use `agent-introspection-debugging` - Code review — use language-specific reviewer agents - Security scanning — use `security-review` or `security-review/scan` - Agent performance benchmarking — use `agent-eval` - Writing new features — use the appropriate workflow skill

## The 12-Layer Stack

Every agent system has these layers. Any of them can corrupt the answer:

| # | Layer | What Goes Wrong | |---|-------|----------------| | 1 | System prompt | Conflicting instructions, instruction bloat | | 2 | Session history | Stale context injection from previous turns | | 3 | Long-term memory | Pollution across sessions, old topics in new conversations | | 4 | Distillation | Compressed artifacts re-entering as pseudo-facts | | 5 | Active recall | Redundant re-summary layers wasting context | | 6 | Tool selection | Wrong tool routing, model skips required tools | | 7 | Tool execution | Hallucinated execution — claims to call but doesn't | | 8 | Tool interpretation | Misread or ignored tool output | | 9 | Answer shaping | Format corruption in final response | | 10 | Platform rendering | Transport-layer mutation (UI, API, CLI mutates valid answers) | | 11 | Hidden repair loops | Silent fallback/retry agents running second LLM pass | | 12 | Persistence | Expired state or cached artifacts reused as live evidence |

## Common Failure Patterns

### 1. Wrapper Regression

The base model produces correct answers, but the wrapper layers make it worse.

**Symptoms:** - Model works fine in playground or direct API call, breaks in your agent - Added a new prompt layer, existing behavior degraded - Agent sounds confident but is confidently wrong - "It was working before the last update"

### 2. Memory Contamination

Old topics leak into new conversations through history, memory retrieval, or distillation.

**Symptoms:** - Agent brings up unrelated past topics - User corrections don't stick (old memory overwrites new) - Same-session artifacts re-enter as pseudo-facts - Memory grows without bound, degrading response quality over time

### 3. Tool Discipline Failure

Tools are declared in the prompt but not enforced in code. The model skips them or hallucinates execution.

**Symptoms:** - "Must use tool X" in prompt, but model answers without calling it - Tool results look correct but were never actually executed - Different tools fight over the same responsibility - Model uses tool when it shouldn't, or skips it when it must

### 4. Rendering/Transport Corruption

The agent's internal answer is correct, but the platform layer mutates it during delivery.

**Symptoms:** - Logs show correct answer, user sees broken output - Markdown rendering, JSON parsing, or streaming fragments corrupt valid responses - Hidden fallback agent quietly replaces the answer before delivery - Output differs between terminal and UI

### 5. Hidden Agent Layers

Silent repair, retry, summarization, or recall agents run without explicit contracts.

**Symptoms:** - Output changes between internal generation and user delivery - "Auto-fix" loops run a second LLM pass the user doesn't know about - Multiple agents modify the same output without coordination - Answers get "smoothed" or "corrected" by invisible layers

## Audit Workflow

### Phase 1: Scope

Define what you're auditing:

- **Target system** — what agent application? - **Entrypoints** — how do users interact with it? - **Model stack** — which LLM(s) and providers? - **Symptoms** — what does the user report? - **Time window** — when did it start? - **Layers to audit** — which of the 12 layers apply?

### Phase 2: Evidence Collection

Gather evidence from the codebase:

- **Source code** — agent loop, tool router, memory admission, prompt assembly - **Logs** — historical session traces, tool call records - **Config** — prompt templates, tool schemas, provider settings - **Memory files** — SOPs, knowledge bases, session archives

Use `rg` to search for anti-patterns:

```bash # Tool requirements expressed only in prompt text (not code) rg "must.*tool|必须.*工具|required.*call" --type md

# Tool execution without validation rg "tool_call|toolCall|tool_use" --type py --type ts

# Hidden LLM calls outside main agent loop rg "completion|chat\.create|messages\.create|llm\.invoke"

# Memory admission without user-correction priority rg "memory.*admit|long.*term.*update|persist.*memory" --type py --type ts

# Fallback loops that run additional LLM calls rg "fallback|retry.*llm|repair.*prompt|re-?prompt" --type py --type ts

# Silent output mutation rg "mutate|rewrite.*response|transform.*output|shap" --type py --type ts ```

### Phase 3: Failure Mapping

For each finding, document:

- **Symptom** — what the user sees - **Mechanism** — how the wrapper causes it - **Source layer** — which of the 12 layers - **Root cause** — the deepest cause - **Evidence** — file:line or log:row reference - **Confidence** — 0.0 to 1.0

### Phase 4: Fix Strategy

Default fix order (code-first, not prompt-first):

1. **Code-gate tool requirements** — enforce in code, not just prompt text 2. **Remove or narrow hidden repair agents** — make fallback explicit with contracts 3. **Reduce context duplication** — same info through prompt + history + memory + distillation 4. **Tighten memory admission** — user corrections > agent assertions 5. **Tighten distillation triggers** — don't compress what shouldn't be compressed 6. **Reduce rendering mutation** — pass-through, don't transform 7. **Convert to typed JSON envelopes** — structured internal flow, not freeform prose

## Severity Model

| Level | Meaning | Action | |-------|---------|--------| | `critical` | Agent can confidently produce wrong operational behavior | Fix before next release | | `high` | Agent frequently degrades correctness or stability | Fix this sprint | | `medium` | Correctness usually survives but output is fragile or wasteful | Plan for next cycle | | `low` | Mostly cosmetic or maintainability issues | Backlog |

## Output Format

Present findings to the user in this order:

1. **Severity-ranked findings** (most critical first) 2. **Architecture diagnosis** (which layer corrupted what, and why) 3. **Ordered fix plan** (code-first, not prompt-first)

Do not lead with compliments or summaries. If the system is broken, say so directly.

## Quick Diagnostic Questions

When auditing an agent system, answer these:

| # | Question | If Yes → | |---|----------|----------| | 1 | Can the model skip a required tool and still answer? | Tool not code-gated | | 2 | Does old conversation content appear in new turns? | Memory contamination | | 3 | Is the same info in system prompt AND memory AND history? | Context duplication | | 4 | Does the platform run a second LLM pass before delivery? | Hidden repair loop | | 5 | Does the output differ between internal generation and user delivery? | Rendering corruption | | 6 | Are "must use tool X" rules only in prompt text? | Tool discipline failure | | 7 | Can the agent's own monologue become persistent memory? | Memory poisoning |

## Anti-Patterns to Avoid

- Avoid blaming the model before falsifying wrapper-layer regressions. - Avoid blaming memory without showing the contamination path. - Do not let a clean current state erase a dirty historical incident. - Do not treat markdown prose as a trustworthy internal protocol. - Do not accept "must use tool" in prompt text when code never enforces it. - Keep findings direct, evidence-backed, and severity-ranked.

## Report Schema

Audits should produce structured reports following this shape:

```json { "schema_version": "ecc.agent-architecture-audit.report.v1", "executive_verdict": { "overall_health": "high_risk", "primary_failure_mode": "string", "most_urgent_fix": "string" }, "scope": { "target_name": "string", "model_stack": ["string"], "layers_to_audit": ["string"] }, "findings": [ { "severity": "critical|high|medium|low", "title": "string", "mechanism": "string", "source_layer": "string", "root_cause": "string", "evidence_refs": ["file:line"], "confidence": 0.0, "recommended_fix": "string" } ], "ordered_fix_plan": [ { "order": 1, "goal": "string", "why_now": "string", "expected_effect": "string" } ] } ```

## Related Skills

- `agent-introspection-debugging` — Debug agent runtime failures (loops, timeouts, state errors) - `agent-eval` — Benchmark agent performance head-to-head - `security-review` — Security audit for code and configuration - `autonomous-agent-harness` — Set up autonomous agent operations - `agent-harness-construction` — Build agent harnesses from scratch

Intended uses

  • Use Agent Architecture Audit when this documented workflow matches the task.

Implementation notes

  • Review the source instructions and adapt inputs before running the workflow.

Try it, then tell us how it went

A copied prompt does not tell us whether the task worked. Report your result after checking the output. We collect your tool and outcome, never your files or prompt.