Skip to main content
Agent skill

Santa Method

Multi-agent adversarial verification: two independent reviewers with the same rubric must both pass before output ships, with a fix-and-re-review convergence loop and human escalation cap. Use when gating publishing, production deploys, compliance or brand-sensitive content, or hallucination-prone claims before they ship.

Get started

Get the complete skill folder, including its scripts and reference files.

·

When to use this skill

Use Santa Method when an AI agent needs a reusable SKILL.md workflow for this job: Multi-agent adversarial verification: two independent reviewers with the same rubric must both pass before output ships, with a fix-and-re-review convergence loop and human escalation cap. Use when gating publishing, production deploys, compliance or brand-sensitive content, or hallucination-prone claims before they ship.

When not to use it

Skip Santa Method when the task is outside the analysis category, or when a more specific skill in this directory already covers the same workflow with clearer triggers.

How to install

Where will you use it?

Install scope
Claude Code folder setup
mkdir -p ~/.claude/skills/santa-method
# Save SKILL.md to ~/.claude/skills/santa-method/SKILL.md
  1. Claude Code personal install: place the skill and its bundled files at ~/.claude/skills/santa-method/SKILL.md. For a shared project, commit the folder at .claude/skills/santa-method/ instead.
  2. Current Claude Code detects SKILL.md changes in watched directories during the session. If you just created a top-level skills directory, run /reload-skills. Check that the skill is listed before testing its trigger.
  3. For Claude chat or Cowork in the Desktop app, manage uploaded skills in Customize > Skills; copying files into a local Claude Code folder is not a Desktop install. Other agents may use different install locations.

Full install guide for Claude, Cursor, and Codex →

Try a starter prompt

After installing the skill, paste this starter prompt into your agent to verify it triggers on a small task.

Starter prompt
Use the Santa Method skill for this task: Use Santa Method when this documented workflow matches the task. Confirm any required inputs or files before making changes.

What supports this recommendation?

71/100 provisional content score. This measures documentation, practical detail and source signals. It is not a measured task success rate or a security certification.

Documentation
77/100
Practical detail
68/100
Source evidence
63/100
Source signals
72/100
Model compatibility evidence
  • claudeinferred

    The skill text mentions Claude or a closely associated term. This is keyword-derived compatibility; no hands-on model test is recorded.

  • chatgptuntested

    No model-specific signal or recorded compatibility test was found.

  • geminiuntested

    No model-specific signal or recorded compatibility test was found.

  • copilotuntested

    No model-specific signal or recorded compatibility test was found.

  • llamauntested

    No model-specific signal or recorded compatibility test was found.

  • perplexityuntested

    No model-specific signal or recorded compatibility test was found.

  • mistraluntested

    No model-specific signal or recorded compatibility test was found.

  • grokuntested

    No model-specific signal or recorded compatibility test was found.

Inferred compatibility is based on content signals. Check the app, its available tools and setup requirements before running a skill.

What this skill does

# Santa Method

Multi-agent adversarial verification framework. Make a list, check it twice. If it's naughty, fix it until it's nice.

The core insight: a single agent reviewing its own output shares the same biases, knowledge gaps, and systematic errors that produced the output. Two independent reviewers with no shared context break this failure mode.

## When to Activate

Invoke this skill when: - Output will be published, deployed, or consumed by end users - Compliance, regulatory, or brand constraints must be enforced - Code ships to production without human review - Content accuracy matters (technical docs, educational material, customer-facing copy) - Batch generation at scale where spot-checking misses systemic patterns - Hallucination risk is elevated (claims, statistics, API references, legal language)

Do NOT use for internal drafts, exploratory research, or tasks with deterministic verification (use build/test/lint pipelines for those).

## Architecture

``` ┌─────────────┐ │ GENERATOR │ Phase 1: Make a List │ (Agent A) │ Produce the deliverable └──────┬───────┘ │ output ▼ ┌──────────────────────────────┐ │ DUAL INDEPENDENT REVIEW │ Phase 2: Check It Twice │ │ │ ┌───────────┐ ┌───────────┐ │ Two agents, same rubric, │ │ Reviewer B │ │ Reviewer C │ │ no shared context │ └─────┬─────┘ └─────┬─────┘ │ │ │ │ │ └────────┼──────────────┼────────┘ │ │ ▼ ▼ ┌──────────────────────────────┐ │ VERDICT GATE │ Phase 3: Naughty or Nice │ │ │ B passes AND C passes → NICE │ Both must pass. │ Otherwise → NAUGHTY │ No exceptions. └──────┬──────────────┬─────────┘ │ │ NICE NAUGHTY │ │ ▼ ▼ [ SHIP ] ┌─────────────┐ │ FIX CYCLE │ Phase 4: Fix Until Nice │ │ │ iteration++ │ Collect all flags. │ if i > MAX: │ Fix all issues. │ escalate │ Re-run both reviewers. │ else: │ Loop until convergence. │ goto Ph.2 │ └──────────────┘ ```

## Phase Details

### Phase 1: Make a List (Generate)

Execute the primary task. No changes to your normal generation workflow. Santa Method is a post-generation verification layer, not a generation strategy.

```python # The generator runs as normal output = generate(task_spec) ```

### Phase 2: Check It Twice (Independent Dual Review)

Spawn two review agents in parallel. Critical invariants:

1. **Context isolation** — neither reviewer sees the other's assessment 2. **Identical rubric** — both receive the same evaluation criteria 3. **Same inputs** — both receive the original spec AND the generated output 4. **Structured output** — each returns a typed verdict, not prose

```python REVIEWER_PROMPT = """ You are an independent quality reviewer. You have NOT seen any other review of this output.

## Task Specification {task_spec}

## Output Under Review {output}

## Evaluation Rubric {rubric}

## Instructions Evaluate the output against EACH rubric criterion. For each: - PASS: criterion fully met, no issues - FAIL: specific issue found (cite the exact problem)

Return your assessment as structured JSON: { "verdict": "PASS" | "FAIL", "checks": [ {"criterion": "...", "result": "PASS|FAIL", "detail": "..."} ], "critical_issues": ["..."], // blockers that must be fixed "suggestions": ["..."] // non-blocking improvements }

Be rigorous. Your job is to find problems, not to approve. """ ```

```python # Spawn reviewers in parallel (Claude Code subagents) review_b = Agent(prompt=REVIEWER_PROMPT.format(...), description="Santa Reviewer B") review_c = Agent(prompt=REVIEWER_PROMPT.format(...), description="Santa Reviewer C")

# Both run concurrently — neither sees the other ```

### Rubric Design

The rubric is the most important input. Vague rubrics produce vague reviews. Every criterion must have an objective pass/fail condition.

| Criterion | Pass Condition | Failure Signal | |-----------|---------------|----------------| | Factual accuracy | All claims verifiable against source material or common knowledge | Invented statistics, wrong version numbers, nonexistent APIs | | Hallucination-free | No fabricated entities, quotes, URLs, or references | Links to pages that don't exist, attributed quotes with no source | | Completeness | Every requirement in the spec is addressed | Missing sections, skipped edge cases, incomplete coverage | | Compliance | Passes all project-specific constraints | Banned terms used, tone violations, regulatory non-compliance | | Internal consistency | No contradictions within the output | Section A says X, section B says not-X | | Technical correctness | Code compiles/runs, algorithms are sound | Syntax errors, logic bugs, wrong complexity claims |

#### Domain-Specific Rubric Extensions

**Content/Marketing:** - Brand voice adherence - SEO requirements met (keyword density, meta tags, structure) - No competitor trademark misuse - CTA present and correctly linked

**Code:** - Type safety (no `any` leaks, proper null handling) - Error handling coverage - Security (no secrets in code, input validation, injection prevention) - Test coverage for new paths

**Compliance-Sensitive (regulated, legal, financial):** - No outcome guarantees or unsubstantiated claims - Required disclaimers present - Approved terminology only - Jurisdiction-appropriate language

### Phase 3: Naughty or Nice (Verdict Gate)

```python def santa_verdict(review_b, review_c): """Both reviewers must pass. No partial credit.""" if review_b.verdict == "PASS" and review_c.verdict == "PASS": return "NICE" # Ship it

# Merge flags from both reviewers, deduplicate all_issues = dedupe(review_b.critical_issues + review_c.critical_issues) all_suggestions = dedupe(review_b.suggestions + review_c.suggestions)

return "NAUGHTY", all_issues, all_suggestions ```

Why both must pass: if only one reviewer catches an issue, that issue is real. The other reviewer's blind spot is exactly the failure mode Santa Method exists to eliminate.

### Phase 4: Fix Until Nice (Convergence Loop)

```python MAX_ITERATIONS = 3

for iteration in range(MAX_ITERATIONS): verdict, issues, suggestions = santa_verdict(review_b, review_c)

if verdict == "NICE": log_santa_result(output, iteration, "passed") return ship(output)

# Fix all critical issues (suggestions are optional) output = fix_agent.execute( output=output, issues=issues, instruction="Fix ONLY the flagged issues. Do not refactor or add unrequested changes." )

# Re-run BOTH reviewers on fixed output (fresh agents, no memory of previous round) review_b = Agent(prompt=REVIEWER_PROMPT.format(output=output, ...)) review_c = Agent(prompt=REVIEWER_PROMPT.format(output=output, ...))

# Exhausted iterations — escalate log_santa_result(output, MAX_ITERATIONS, "escalated") escalate_to_human(output, issues) ```

Critical: each review round uses **fresh agents**. Reviewers must not carry memory from previous rounds, as prior context creates anchoring bias.

## Implementation Patterns

### Pattern A: Claude Code Subagents (Recommended)

Subagents provide true context isolation. Each reviewer is a separate process with no shared state.

```bash # In a Claude Code session, use the Agent tool to spawn reviewers # Both agents run in parallel for speed ```

```python # Pseudocode for Agent tool invocation reviewer_b = Agent( description="Santa Review B", prompt=f"Review this output for quality...\n\nRUBRIC:\n{rubric}\n\nOUTPUT:\n{output}" ) reviewer_c = Agent( description="Santa Review C", prompt=f"Review this output for quality...\n\nRUBRIC:\n{rubric}\n\nOUTPUT:\n{output}" ) ```

### Pattern B: Sequential Inline (Fallback)

When subagents aren't available, simulate isolation with explicit context resets:

1. Generate output 2. New context: "You are Reviewer 1. Evaluate ONLY against this rubric. Find problems." 3. Record findings verbatim 4. Clear context completely 5. New context: "You are Reviewer 2. Evaluate ONLY against this rubric. Find problems." 6. Compare both reviews, fix, repeat

The subagent pattern is strictly superior — inline simulation risks context bleed between reviewers.

### Pattern C: Batch Sampling

For large batches (100+ items), full Santa on every item is cost-prohibitive. Use stratified sampling:

1. Run Santa on a random sample (10-15% of batch, minimum 5 items) 2. Categorize failures by type (hallucination, compliance, completeness, etc.) 3. If systematic patterns emerge, apply targeted fixes to the entire batch 4. Re-sample and re-verify the fixed batch 5. Continue until a clean sample passes

```python import random

def santa_batch(items, rubric, sample_rate=0.15): sample = random.sample(items, max(5, int(len(items) * sample_rate)))

for item in sample: result = santa_full(item, rubric) if result.verdict == "NAUGHTY": pattern = classify_failure(result.issues) items = batch_fix(items, pattern) # Fix all items matching pattern return santa_batch(items, rubric) # Re-sample

return items # Clean sample → ship batch ```

## Failure Modes and Mitigations

| Failure Mode | Symptom | Mitigation | |-------------|---------|------------| | Infinite loop | Reviewers keep finding new issues after fixes | Max iteration cap (3). Escalate. | | Rubber stamping | Both reviewers pass everything | Adversarial prompt: "Your job is to find problems, not approve." | | Subjective drift | Reviewers flag style preferences, not errors | Tight rubric with objective pass/fail criteria only | | Fix regression | Fixing issue A introduces issue B | Fresh reviewers each round catch regressions | | Reviewer agreement bias | Both reviewers miss the same thing | Mitigated by independence, not eliminated. For critical output, add a third reviewer or human spot-check. | | Cost explosion | Too many iterations on large outputs | Batch sampling pattern. Budget caps per verification cycle. |

## Integration with Other Skills

| Skill | Relationship | |-------|-------------| | Verification Loop | Use for deterministic checks (build, lint, test). Santa for semantic checks (accuracy, hallucinations). Run verification-loop first, Santa second. | | Eval Harness | Santa Method results feed eval metrics. Track pass@k across Santa runs to measure generator quality over time. | | Continuous Learning v2 | Santa findings become instincts. Repeated failures on the same criterion → learned behavior to avoid the pattern. | | Strategic Compact | Run Santa BEFORE compacting. Don't lose review context mid-verification. |

## Metrics

Track these to measure Santa Method effectiveness:

- **First-pass rate**: % of outputs that pass Santa on round 1 (target: >70%) - **Mean iterations to convergence**: average rounds to NICE (target: <1.5) - **Issue taxonomy**: distribution of failure types (hallucination vs. completeness vs. compliance) - **Reviewer agreement**: % of issues flagged by both reviewers vs. only one (low agreement = rubric needs tightening) - **Escape rate**: issues found post-ship that Santa should have caught (target: 0)

## Cost Analysis

Santa Method costs approximately 2-3x the token cost of generation alone per verification cycle. For most high-stakes output, this is a bargain:

``` Cost of Santa = (generation tokens) + 2×(review tokens per round) × (avg rounds) Cost of NOT Santa = (reputation damage) + (correction effort) + (trust erosion) ```

For batch operations, the sampling pattern reduces cost to ~15-20% of full verification while catching >90% of systematic issues.

Intended uses

  • Use Santa Method when this documented workflow matches the task.

Implementation notes

  • Review the source instructions and adapt inputs before running the workflow.

Try it, then tell us how it went

A copied prompt does not tell us whether the task worked. Report your result after checking the output. We collect your tool and outcome, never your files or prompt.