Skip to main content
Agent skill

Bend Harness Qa

Drive a live bend-harness session programmatically with bendclient.py to test features end to end, reproduce and investigate bugs, and write precise bug reports. Use when asked to test, verify, QA, benchmark, or investigate the Bend harness.

Get started

Get the complete skill folder, including its scripts and reference files.

When to use this skill

Use Bend Harness Qa when an AI agent needs a reusable SKILL.md workflow for this job: Drive a live bend-harness session programmatically with bendclient.py to test features end to end, reproduce and investigate bugs, and write precise bug reports. Use when asked to test, verify, QA, benchmark, or investigate the Bend harness.

When not to use it

Skip Bend Harness Qa when the task is outside the coding category, or when a more specific skill in this directory already covers the same workflow with clearer triggers.

How to install

Where will you use it?

Install scope
Claude Code folder setup
mkdir -p ~/.claude/skills/bend-harness-qa
# Save SKILL.md to ~/.claude/skills/bend-harness-qa/SKILL.md
  1. Claude Code personal install: place the skill and its bundled files at ~/.claude/skills/bend-harness-qa/SKILL.md. For a shared project, commit the folder at .claude/skills/bend-harness-qa/ instead.
  2. Current Claude Code detects SKILL.md changes in watched directories during the session. If you just created a top-level skills directory, run /reload-skills. Check that the skill is listed before testing its trigger.
  3. For Claude chat or Cowork in the Desktop app, manage uploaded skills in Customize > Skills; copying files into a local Claude Code folder is not a Desktop install. Other agents may use different install locations.

Full install guide for Claude, Cursor, and Codex →

Try a starter prompt

After installing the skill, paste this starter prompt into your agent to verify it triggers on a small task.

Starter prompt
Use the Bend Harness Qa skill for this task: Use Bend Harness Qa when this documented workflow matches the task. Confirm any required inputs or files before making changes.

What supports this recommendation?

70/100 provisional content score. This measures documentation, practical detail and source signals. It is not a measured task success rate or a security certification.

Documentation
77/100
Practical detail
68/100
Source evidence
57/100
Source signals
72/100
Model compatibility evidence
  • claudeuntested

    No model-specific signal or recorded compatibility test was found.

  • chatgptuntested

    No model-specific signal or recorded compatibility test was found.

  • geminiuntested

    No model-specific signal or recorded compatibility test was found.

  • copilotuntested

    No model-specific signal or recorded compatibility test was found.

  • llamauntested

    No model-specific signal or recorded compatibility test was found.

  • perplexityuntested

    No model-specific signal or recorded compatibility test was found.

  • mistraluntested

    No model-specific signal or recorded compatibility test was found.

  • grokuntested

    No model-specific signal or recorded compatibility test was found.

Inferred compatibility is based on content signals. Check the app, its available tools and setup requirements before running a skill.

What this skill does

# Bend Harness QA

You control a REAL live harness session (GLM model + bash + MCP tools + skills) from Python: the same wire protocol the TUI speaks, but scripted and reproducible. This is how you test a new feature end to end, reproduce a user-reported bug, or investigate weird behavior.

Repo: `/Users/gabrielvergnaud/lab/bend-lab/harness` (all relative paths below are inside it).

## Golden rules

1. ALWAYS run with `python3 -u` (unbuffered stdout) — without it a long turn looks dead. 2. One TCP connection per session: the REPL serves one client at a time. Sequential `BendSession.fresh()` calls are fine (each picks a private port); always `s.close()` when done (the REPL dies with it, the session checkpoint stays). 3. A turn ends at the `--- idle` line. `say()` blocks until then (default timeout 900s — pass a shorter one for smoke tests). 4. Bash tool calls slower than `BEND_BG_AFTER` seconds (default 30) hand off to a background slot instead of blocking. Use `fresh(bg_after=N)` to shrink the window in tests.

## Minimal run

```python import sys; sys.path.insert(0, "scripts") # run from the repo root from bend_client import BendSession

s = BendSession.fresh() # or BendSession.resume(path) try: lines = s.say("reply with exactly OK", timeout=120) assert "OK" in s.last_assistant(lines) print("PASS") finally: s.close() ```

Run it: `python3 -u /tmp/mytest.py`. The full turn transcript is in `lines` (print it when debugging); the client log is `/tmp/bend-client-<port>.log`.

For anything beyond the basics, read `reference/client-api.md` (full client API + the wire events) and copy `scripts/e2e_template.py` as a starting point.

## E2E recipes

- **Feature smoke test**: one turn, instruct precisely, then assert on the obs lines (tool started/finished, tool_result) AND `last_assistant()`. Print PASS/FAIL, never eyeball. - **Mid-turn steering**: run `say()` in a thread, wait for the tool call to appear in the log, then `s.steer_midturn(text)` — it writes the file side-channel `/tmp/bend-steer-<port>.txt`. Assert: an obs `steering_received:` mid-turn, an obs `steered:` (injected into the next model input), and the final answer honoring the steered text. (`s.steer()` sends on the SOCKET — it is only read at idle and starts a NEW turn; that is NOT mid-turn steering.) - **Background commands**: `BendSession.fresh(bg_after=2)`, ask the agent to run `sleep 5 && echo done`. Assert the tool result carries the handoff contract (slot id, tail/stdin/status/kill commands) and that a second bash call can read the slot's `.out`/`.rc` under `/tmp/bend-bg-<port>/`. - **Skills**: ask the model to call the `skill` tool with exact args `{"name":"<exact>"}`. Assert the result starts with the SKILL.md CONTENT (frontmatter stripped), and that a bogus name returns "Skill ... is not available" plus the available list. - **Sessions / CLI flags** (not client-driven): `./run.sh --headless [--scripted]` (what bend_client.py runs: one READY line, lives until stdin closes; there is no single-agent TUI any more, BISE-113). Sessions live in `~/.bend-harness/sessions/<id>.txt`; `--continue` = latest mtime, `--resume <id-or-unique-prefix>`, errors list the available ids.

## Before you report a bug

Separate a harness bug from an environment/model flake — run the deterministic gates from the repo root:

```sh python3 tests/proof_run.py . 4 "$TMPDIR/proofs" # laws: every bend/proofs/*.bend (NEVER `bend update`: pin 2.0.27) bend bend/runtime/demo.bend -o harness-demo && ./harness-demo > /tmp/demo-a.txt && ./harness-demo > /tmp/demo-b.txt && diff /tmp/demo-a.txt /tmp/demo-b.txt # deterministic ~/.bend/bin/bend bend/tests/test-bg.bend -o /tmp/bg-suite && rm -rf /tmp/bgtest && BEND_BG_ROOT=/tmp/bgtest BEND_BG_AFTER=2 /tmp/bg-suite ```

A live-model failure with all gates green is a live-integration bug or a model flake — re-run the e2e once before reporting.

## Bug report format

Write reports as markdown (scratchpad or the file the user asked for), one report per bug:

1. **Title** — one line, the symptom. 2. **Environment** — commit hash (`git log --oneline -1`), mode (live/scripted), env vars (BEND_BG_AFTER, port). 3. **Repro** — the SMALLEST python script that triggers it (must be runnable with `python3 -u`), or the exact pipe command. 4. **Expected / Observed** — two short lines, not prose. 5. **Transcript excerpt** — the relevant obs lines only, not the whole turn. 6. **Hypothesis** — the suspected module (`runtime/main.bend`, `runtime/bash.bend`, `runtime/provider.bend`, `core/session.bend`, the TUI...) and what the gates say. 7. **Severity** — blocks-turn / degrades / cosmetic.

## Investigation pointers

- Session checkpoints: `/tmp/bend-sessions/session-<port>.txt` (client-spawned) or `~/.bend-harness/sessions/` (binary-spawned). Format: the checkpoint text protocol (core/checkpoint.bend). - The harness can test ITSELF: via `say()`, ask the live agent to exercise its own tools (bash, background slots, skills, programmatic calling) and return a structured report — it knows its own contract. That is the QA-round pattern; parse its final answer for findings. - Tool routing: `bend/runtime/main.bend` (`exec_pick` top-level, `exec_program.call` from the sandbox). Pure contracts are pinned in `bend/laws/*.bend`, each closed by its `bend/proofs/*.bend`. - MCP index `~/.bend-harness/mcp-index.txt`, skills index `~/.bend-harness/skills-index.txt`.

## Gotchas

- No `timeout` command on macOS (GNU coreutils absent) — use the client timeout args. - Assistant text escapes newlines as literal `\n` on the wire — read it via `last_assistant()`/`unescape()`. - The client log shows `REPL on` when the REPL is ready; a TCP probe would steal the resume greeting — never probe, wait for the banner. - A client-spawned session is LIVE (real model, real bash): commands the agent runs are real commands on this machine.

Intended uses

  • Use Bend Harness Qa when this documented workflow matches the task.

Implementation notes

  • Review the source instructions and adapt inputs before running the workflow.

Try it, then tell us how it went

A copied prompt does not tell us whether the task worked. Report your result after checking the output. We collect your tool and outcome, never your files or prompt.