Skip to main content
Agent skill

Openclaw Test Performance

Benchmark, diagnose, and optimize OpenClaw test and plugin-suite runtime, import hotspots, CPU/RSS, heap growth, and slow coverage paths.

Get started

Get the complete skill folder, including its scripts and reference files.

When to use this skill

Use Openclaw Test Performance when an AI agent needs a reusable SKILL.md workflow for this job: Benchmark, diagnose, and optimize OpenClaw test and plugin-suite runtime, import hotspots, CPU/RSS, heap growth, and slow coverage paths.

When not to use it

Skip Openclaw Test Performance when the task is outside the coding category, or when a more specific skill in this directory already covers the same workflow with clearer triggers.

How to install

Where will you use it?

Install scope
Claude Code folder setup
mkdir -p ~/.claude/skills/openclaw-test-performance
# Save SKILL.md to ~/.claude/skills/openclaw-test-performance/SKILL.md
  1. Claude Code personal install: place the skill and its bundled files at ~/.claude/skills/openclaw-test-performance/SKILL.md. For a shared project, commit the folder at .claude/skills/openclaw-test-performance/ instead.
  2. Current Claude Code detects SKILL.md changes in watched directories during the session. If you just created a top-level skills directory, run /reload-skills. Check that the skill is listed before testing its trigger.
  3. For Claude chat or Cowork in the Desktop app, manage uploaded skills in Customize > Skills; copying files into a local Claude Code folder is not a Desktop install. Other agents may use different install locations.

Full install guide for Claude, Cursor, and Codex →

Try a starter prompt

After installing the skill, paste this starter prompt into your agent to verify it triggers on a small task.

Starter prompt
Use the Openclaw Test Performance skill for this task: Read the relevant local AGENTS.md files before editing:. Confirm any required inputs or files before making changes.

What supports this recommendation?

82/100 provisional content score. This measures documentation, practical detail and source signals. It is not a measured task success rate or a security certification.

Documentation
89/100
Practical detail
92/100
Source evidence
63/100
Source signals
72/100
Model compatibility evidence
  • claudeuntested

    No model-specific signal or recorded compatibility test was found.

  • chatgptuntested

    No model-specific signal or recorded compatibility test was found.

  • geminiuntested

    No model-specific signal or recorded compatibility test was found.

  • copilotuntested

    No model-specific signal or recorded compatibility test was found.

  • llamauntested

    No model-specific signal or recorded compatibility test was found.

  • perplexityuntested

    No model-specific signal or recorded compatibility test was found.

  • mistraluntested

    No model-specific signal or recorded compatibility test was found.

  • grokuntested

    No model-specific signal or recorded compatibility test was found.

Inferred compatibility is based on content signals. Check the app, its available tools and setup requirements before running a skill.

What this skill does

# OpenClaw Test Performance

Use evidence first. The goal is real `pnpm test`, plugin-suite, and plugin-inspector speed/RSS improvement with coverage intact, not runner tuning by guesswork.

## Workflow

1. Read the relevant local `AGENTS.md` files before editing: - `src/agents/AGENTS.md` for agent/import hotspots. - `src/channels/AGENTS.md` and `src/plugins/AGENTS.md` for plugin/channel laziness. - `src/gateway/AGENTS.md` for server lifecycle tests. - `test/helpers/AGENTS.md` and `src/channels/plugins/contracts/test-helpers/AGENTS.md` for shared contract helpers. - `src/infra/outbound/AGENTS.md` for outbound/media/action tests. 2. Establish a baseline before changing code: - Prefer `pnpm test:perf:groups --full-suite --allow-failures --output <file>` for full-suite ranking. - For bundled plugin breadth, run the smallest relevant `pnpm test:extensions:batch <plugin[,plugin...]>` or plugin-inspector command before jumping to the full extension sweep. - For a scoped hotspot use: `/usr/bin/time -l pnpm test <file-or-files> --maxWorkers=1 --reporter=verbose` - For import-heavy suspicion add: `OPENCLAW_VITEST_IMPORT_DURATIONS=1 OPENCLAW_VITEST_PRINT_IMPORT_BREAKDOWN=1`. 3. Separate wall/runner noise from real file cost: - Compare Vitest duration, test body timing, import breakdown, wall time, and max RSS. - Re-run single files when grouped/full-suite numbers look stale or noisy. - If a full-suite grouped run reports a lane failure but JSON says tests passed, capture that as harness/noise and verify the suspect file directly. 4. Pick the next attack by return and risk: - High return: one file/test dominates seconds or RSS and has a clear root. - High leverage: one plugin or SDK barrel causes every plugin-inspector or extension-batch run to load broad runtime. - Lower risk: static descriptors, target parsing, routing, auth bypass, setup hints, registry fixtures, or test server lifecycle. - Higher risk: real memory/runtime behavior, live providers, protocol contracts, or broad production refactors. 5. Fix the root cause, not the symptom: - Move static metadata/parsing into narrow helpers or lightweight artifacts reused by full runtime and fast paths. - Prefer dependency injection, loaded-plugin-only lookup, explicit fixtures, and pure helpers over broad mocks. - Reuse suite-level servers/clients when a fresh handshake is irrelevant. - Keep schedulers/background loops off unless the test proves scheduling. - In plugin paths, move static metadata into manifest/lightweight artifacts and keep runtime plugin loads behind explicit execution boundaries. 6. Preserve coverage shape: - Do not delete a slow integration proof unless the exact production composition is extracted into a named helper and tested. - Keep one cheap integration smoke when cross-component wiring matters. - State explicitly what incidental coverage was removed, if any. 7. Re-benchmark the same command after the change and compute seconds plus percent gain. 8. Update the running report when requested or when this thread is tracking one. Include before/after commands, artifacts, coverage notes, verification, and next attack order. 9. Stage the intended paths, commit with standard Git, and push when the user asked for commits/pushes. Stage only files touched for this attack.

## Plugin-Suite Workflow

Use this section when perf work involves bundled plugins, plugin-inspector, SDK barrels, package-boundary tests, or extension suites.

1. Map the suite shape first: - source tests: `pnpm test extensions/<id>` or `pnpm test:extensions:batch <id>` - package boundaries: `pnpm run test:extensions:package-boundary:canary` and `pnpm run test:extensions:package-boundary:compile` - all bundled source tests: `pnpm test:extensions` - plugin import memory: `pnpm test:extensions:memory -- --json .artifacts/test-perf/extensions-memory.json` - plugin-inspector/report work: keep report primitives in `plugin-inspector`; keep wrappers thin and collect peak RSS when the command supports it. 2. Start narrow, then widen: - one plugin changed: run that plugin's tests and plugin-inspector slice. - SDK/public barrel changed: add representative provider, channel, memory, and feature plugins. - loader/runtime mirror changed: add package-boundary checks and build/package proof as needed. - unknown shared plugin behavior: run `test:extensions:batch` groups before `pnpm test:extensions`. 3. Treat plugin-inspector failures as product signals: - JSON must parse. - warnings/errors must be classified, not hidden. - runtime capture should be quiet and config-tolerant. - command output should include wall time, exit code, and peak RSS when available. 4. Follow `$openclaw-testing` for host selection. Trusted source benchmarks can run locally with comparable machine/load conditions. Use `$crabbox` when clean packaging, Linux/platform behavior, isolation, or an explicit remote request is part of the proof; reuse and clean up only the owned lease. 5. If plugin performance is package-artifact sensitive, switch to `release-openclaw-plugin-testing` and Package Acceptance rather than trusting source-only timing.

## Metric Collection

Collect at least one stable metric before and after. Prefer the same machine and same command. For Testbox comparisons, use the same `tbx_...` id when possible.

| Metric | Use for | Preferred source | | --------------- | ---------------------------------- | --------------------------------------------------------------------------- | | wall time | user-visible suite cost | `/usr/bin/time -l`, test wrapper duration, Testbox run time | | Vitest duration | test body/import cost | Vitest output per file/shard | | import duration | broad barrel/runtime loads | `OPENCLAW_VITEST_IMPORT_DURATIONS=1` | | max RSS | memory pressure and OOM risk | `/usr/bin/time -l`, `pnpm test:extensions:memory`, wrapper memory summaries | | CPU/user/sys | CPU-bound vs wait-bound split | `/usr/bin/time -l` locally, Testbox job timing when local CPU is noisy | | heap evidence | real leak vs retained module graph | `openclaw-test-heap-leaks` workflow |

Local scoped command with CPU/RSS:

```bash timeout 240 /usr/bin/time -l pnpm test <file> --maxWorkers=1 --reporter=verbose ```

Plugin import memory profile:

```bash pnpm build pnpm test:extensions:memory -- --top 20 --json .artifacts/test-perf/extensions-memory.json ```

Targeted plugin import memory:

```bash pnpm test:extensions:memory -- --extension discord --extension telegram --skip-combined ```

Heap/RSS escalation:

```bash pnpm test:perf:groups \ --config test/vitest/vitest.unit-fast.config.ts \ --allow-failures \ --output .artifacts/test-perf/unit-fast-memory.json pnpm test:perf:profile:runner -- \ --output-dir .artifacts/test-perf/vitest-runner-profile -- <file> ```

Use `openclaw-test-heap-leaks` when RSS keeps growing across intervals, workers OOM, or the suspect command has app-object retention. Do not call RSS growth a leak until snapshots or retainers support it.

## Common Root Causes

- Full bundled channel/plugin runtime loaded for static data. - `getChannelPlugin()` fallback used when an already-loaded fixture or pure parser would suffice. - Broad `api.ts`, `runtime-api.ts`, `test-api.ts`, or plugin-sdk barrels pulled into hot tests. - SDK root aliases or package barrels pulling focused subpaths back into a broad plugin graph. - Plugin-inspector loading runtime code just to render metadata, reports, or CI policy scores. - Bundled plugin capture reusing real config/home state instead of synthetic, redacted, isolated state. - Partial-real mocks using `importActual()` around broad modules. - `vi.resetModules()` plus fresh imports in per-test loops. - Test plugin registry seeded in `beforeAll` while runtime state resets in `afterEach`. - Per-test gateway/server/client startup when state reset would suffice. - Runtime/default model/auth selection paid by idle snapshots or fixtures. - Plugin-owned media/action discovery triggered before checking whether args contain plugin-owned fields. - Parallel Vitest runs sharing `node_modules/.experimental-vitest-cache` without distinct `OPENCLAW_VITEST_FS_MODULE_CACHE_PATH` values.

## Benchmark Commands

Scoped file:

```bash timeout 240 /usr/bin/time -l pnpm test <file> --maxWorkers=1 --reporter=verbose ```

Scoped file with import breakdown:

```bash timeout 240 /usr/bin/time -l env \ OPENCLAW_VITEST_IMPORT_DURATIONS=1 \ OPENCLAW_VITEST_PRINT_IMPORT_BREAKDOWN=1 \ pnpm test <file> --maxWorkers=1 --reporter=verbose ```

Grouped suite:

```bash pnpm test:perf:groups --full-suite --allow-failures \ --output .artifacts/test-perf/<name>.json ```

Extension batch:

```bash pnpm test:extensions:batch <plugin[,plugin...]> -- --reporter=verbose ```

All extension tests:

```bash pnpm test:extensions ```

Package-boundary plugin checks:

```bash pnpm run test:extensions:package-boundary:canary pnpm run test:extensions:package-boundary:compile ```

Reuse an existing Vitest JSON report:

```bash pnpm test:perf:groups --report <vitest-json> \ --output .artifacts/test-perf/<name>.json ```

## Verification

- Always run the targeted test surface that proves the change. - Source changes: focused local tests + targeted typecheck/lint/format before push; `pnpm check:changed` optional. PR CI = broad proof. Testbox only for CI/local gaps, immediate grant; warmed Testbox reuse when selected. - For test-only changes, run `pnpm test:changed` or the exact edited tests. - Run `pnpm build` when touching lazy-loading, bundled artifacts, package boundaries, dynamic imports, build output, or public surfaces. - For plugin SDK/barrel/runtime changes, compare exact commits with `pnpm plugin-sdk:api:diff -- --base <base-sha> --head <head-sha>` when the public API surface may drift. For PR-local proof, use the branch merge base as `<base-sha>` and the exact tested head commit as `<head-sha>`. - For plugin-suite perf fixes, verify at least one representative plugin batch plus the changed gate; use Package Acceptance if the bug only exists in a packed artifact. - If deps are missing/stale, run `pnpm install` and retry the exact failed command once. - Use the report format:

```markdown | Metric | Before | After | Gain | | -------------- | -----: | -----: | ------------: | | File wall time | `Xs` | `Ys` | `-Zs` (`P%`) | | Max RSS | `XMB` | `YMB` | `-ZMB` (`P%`) | | CPU user/sys | `X/Ys` | `A/Bs` | explain | ```

## Handoff

Keep the final concise:

- Root cause. - Suite/plugin scope. - Files changed. - Before/after wall, Vitest/import, CPU, and RSS numbers where available. - Leak classification if memory was involved: real leak, retained module graph, or inconclusive. - Coverage retained. - Verification commands. - Testbox ID or workflow URL for remote proof. - Commit hash and push status.

Intended uses

  • Read the relevant local AGENTS.md files before editing:
  • src/agents/AGENTS.md for agent/import hotspots.
  • src/channels/AGENTS.md and src/plugins/AGENTS.md for plugin/channel
  • src/gateway/AGENTS.md for server lifecycle tests.
  • test/helpers/AGENTS.md and
  • src/infra/outbound/AGENTS.md for outbound/media/action tests.
  • Establish a baseline before changing code:
  • Prefer pnpm test:perf:groups --full-suite --allow-failures --output <file>
  • For bundled plugin breadth, run the smallest relevant pnpm
  • For a scoped hotspot use:
  • For import-heavy suspicion add:
  • Separate wall/runner noise from real file cost:
  • Compare Vitest duration, test body timing, import breakdown, wall time, and
  • Re-run single files when grouped/full-suite numbers look stale or noisy.
  • If a full-suite grouped run reports a lane failure but JSON says tests
  • Pick the next attack by return and risk:
  • High return: one file/test dominates seconds or RSS and has a clear root.
  • High leverage: one plugin or SDK barrel causes every plugin-inspector or
  • Lower risk: static descriptors, target parsing, routing, auth bypass,
  • Higher risk: real memory/runtime behavior, live providers, protocol

Implementation notes

  • Review the source instructions and adapt inputs before running the workflow.

Try it, then tell us how it went

A copied prompt does not tell us whether the task worked. Report your result after checking the output. We collect your tool and outcome, never your files or prompt.