Penguin Sdk
Use to build a standalone external program with an agent in it, or connect an external program to a Penguin agent. For an app inside PenguinHarness with its own Agent and workbench tab, use agent-initialization and penguin-plugin directly; no SDK/API question applies. A program reaches an agent in one of two ways, and the skill asks the user which before building — embedded with the Penguin Harness SDK (@prismshadow/penguin-core, no server), or over the Penguin server's Agent API (AMSP, with the @prismshadow/amsp client or curl). This is writing application code, not configuring an Agent State inside PenguinHarness. Covers self-contained projects, the createSession/run streaming loop with thinking and image messages, wiring the user's existing tools in as CLI commands, a complete RAG recipe that ingests documents into a knowledge base and answers with citations behind a web UI, and the Agent API client. Workbenches — the tabs and pages beside the chat that an Agent keeps for itself ins
Get the complete skill folder, including its scripts and reference files.
When to use this skill
Use Penguin Sdk when an AI agent needs a reusable SKILL.md workflow for this job: Use to build a standalone external program with an agent in it, or connect an external program to a Penguin agent. For an app inside PenguinHarness with its own Agent and workbench tab, use agent-initialization and penguin-plugin directly; no SDK/API question applies. A program reaches an agent in one of two ways, and the skill asks the user which before building — embedded with the Penguin Harness SDK (@prismshadow/penguin-core, no server), or over the Penguin server's Agent API (AMSP, with the @prismshadow/amsp client or curl). This is writing application code, not configuring an Agent State inside PenguinHarness. Covers self-contained projects, the createSession/run streaming loop with thinking and image messages, wiring the user's existing tools in as CLI commands, a complete RAG recipe that ingests documents into a knowledge base and answers with citations behind a web UI, and the Agent API client. Workbenches — the tabs and pages beside the chat that an Agent keeps for itself ins
When not to use it
Skip Penguin Sdk when the task is outside the coding category, or when a more specific skill in this directory already covers the same workflow with clearer triggers.
How to install
Where will you use it?
mkdir -p ~/.claude/skills/penguin-sdk # Save SKILL.md to ~/.claude/skills/penguin-sdk/SKILL.md
- Claude Code personal install: place the skill and its bundled files at ~/.claude/skills/penguin-sdk/SKILL.md. For a shared project, commit the folder at .claude/skills/penguin-sdk/ instead.
- Current Claude Code detects SKILL.md changes in watched directories during the session. If you just created a top-level skills directory, run /reload-skills. Check that the skill is listed before testing its trigger.
- For Claude chat or Cowork in the Desktop app, manage uploaded skills in Customize > Skills; copying files into a local Claude Code folder is not a Desktop install. Other agents may use different install locations.
Full install guide for Claude, Cursor, and Codex →
Try a starter prompt
After installing the skill, paste this starter prompt into your agent to verify it triggers on a small task.
Use the Penguin Sdk skill for this task: Use Penguin Sdk when this documented workflow matches the task. Confirm any required inputs or files before making changes.
What supports this recommendation?
70/100 provisional content score. This measures documentation, practical detail and source signals. It is not a measured task success rate or a security certification.
- Documentation
- 77/100
- Practical detail
- 68/100
- Source evidence
- 63/100
- Source signals
- 70/100
Model compatibility evidence
- claudeinferred
The skill text mentions Claude or a closely associated term. This is keyword-derived compatibility; no hands-on model test is recorded.
- chatgptinferred
The skill text mentions ChatGPT or a closely associated term. This is keyword-derived compatibility; no hands-on model test is recorded.
- geminiinferred
The skill text mentions Gemini or a closely associated term. This is keyword-derived compatibility; no hands-on model test is recorded.
- copilotinferred
The skill uses model-agnostic prompt or LLM terminology. Portability is inferred and has not been verified on this model.
- llamainferred
The skill uses model-agnostic prompt or LLM terminology. Portability is inferred and has not been verified on this model.
- perplexityinferred
The skill uses model-agnostic prompt or LLM terminology. Portability is inferred and has not been verified on this model.
- mistralinferred
The skill uses model-agnostic prompt or LLM terminology. Portability is inferred and has not been verified on this model.
- grokinferred
The skill uses model-agnostic prompt or LLM terminology. Portability is inferred and has not been verified on this model.
Inferred compatibility is based on content signals. Check the app, its available tools and setup requirements before running a skill.
What this skill does
# Penguin Harness SDK and Agent API
This skill builds programs that use a Penguin agent. A program reaches one in either of two ways, embedded with the SDK or through the Penguin server's Agent API, and the user picks which. Workbenches — pages and server code an Agent keeps inside PenguinHarness — are a third thing, and the penguin-plugin skill's.
To have an agent perform a task for you, use the `run_subagent` tool: both ways are for programs, not for handing off your own work.
## Two ways to reach an agent
**The SDK: `@prismshadow/penguin-core`, embedded in the program.**
- *What it is.* The agent loop as a TypeScript library. The program creates its own Agent from a data directory inside the project and runs its Sessions in-process. No Penguin server is involved, at build time or at run time. - *What you get.* The basic agent loop: the Agent State files (`AGENTS.md`, Skills, tools), the streamed OmniMessages and an `approve` callback for every tool call, in a self-contained folder that runs wherever Node 24 does, on the program's own model key. - *What you don't get.* No server-managed Sessions (the program keeps its own), no visibility in the Web App, no approvals UI (the program answers in code) and no cost center. The embedded agent belongs to the program: changes to an Agent inside PenguinHarness never reach it. - *Choose it when* the program must stand alone: shipped to other machines or people, run where no Penguin server is, or with the agent built into the product.
**The API: Penguin's Agent API, over AMSP.**
- *What it is.* The program calls an Agent that lives in PenguinHarness, over HTTP. The Penguin server runs each turn and streams it back as AMSP events, and the program continues a conversation by its Session id. TypeScript uses the `@prismshadow/amsp` client (`AgentClient`); any other language, or curl, speaks HTTP and SSE directly. - *What you get.* The Agent as PenguinHarness runs it: its `AGENTS.md`, Skills and memory, on the Project's models and credentials, so the program holds no model key. The server keeps the conversations as archived chats (**Settings › Archived chats** in the Web App), where a person finds one and unarchives it to read it and answer its approvals, and their usage lands in the cost center. - *What it requires.* A running Penguin server that the program can reach over the network; API access turned on for that Agent by the Project's owner, on the Agent's **API** tab; and a key created there, shown once. You never turn these on yourself (see *The API path*). - *Choose it when* the Agent already exists in PenguinHarness or should be managed there, people need to see its conversations or costs, or the program is not written in TypeScript.
### Ask which way first
Whenever the user asks to connect a program to an agent, or to build a program with an agent in it (an AI, agentic or RAG app, a bot, a service, a script), ask which way before you write anything, even when the request is otherwise concrete. It is the only question a concrete request gets: it overrides the no-follow-up-questions default of *Before you start* for this one question.
Do not ask when the way is already chosen:
- in the user's message: "use the SDK", "standalone", "without the Penguin server" or "embed the agent" chooses the SDK; "call it over the API", "through the Agent API", "with AMSP", "with `@prismshadow/amsp`" or "with curl" chooses the API; - earlier in this conversation, for the same program.
Naming this skill does not choose a way. Workbenches (the penguin-plugin skill) take neither way and need no question; neither does connecting a Feishu, Telegram, QQ or WeChat bot to a conversation, which is the Web App's Remote control.
Ask with one sentence and this choice block, in the user's language, as the end of your reply. Mark one option `"recommended": true` only when the request leans that way: an Agent the user already has in PenguinHarness leans to the API, an app to ship to others leans to the SDK.
````markdown Your program can reach the agent in two ways: embedded with the SDK, or through the Penguin server's Agent API.
```a2ui { "type": "choice", "id": "agent-access", "question": "How should the program reach the agent?", "options": [ { "label": "SDK: embed the agent", "value": "Use the SDK: embed the agent in the program", "description": "Runs inside the program on its own model key. No Penguin server; nothing appears in the Web App." }, { "label": "API: call a Penguin Agent", "value": "Use the Agent API: call a Penguin Agent over AMSP", "description": "Calls an Agent on your Penguin server. You turn on its API access and create a key; its conversations appear in the Web App." } ] } ``` ````
The Web App shows the block as two buttons; the CLI and the messaging channels show it as a numbered list. The answer arrives as the user's next message, in plain text: read it like any other message and build on the way it names.
## Before you start
If the user's message only invokes this skill (e.g. "use penguin-sdk skill") without a concrete app to build, ask the user what they want to build. Once the request is concrete, settle the way first (see *Ask which way first*). Then do **not** ask follow-up questions: build it end to end with that way's defaults and list the assumptions you made in your final reply.
- SDK path: a self-contained workspace project, the project default model, BM25 retrieval, a web UI styled per the web-design skill. - API path: a self-contained workspace project in TypeScript on `@prismshadow/amsp` (plain HTTP when the user's program is in another language), calling the Agent the user named ("you" means this Agent) or, with none named, a new Agent created for the purpose with the agent-initialization skill.
Each path also needs one thing only the user can provide: a model credential on the SDK path (see *Keys and the data root*), and on the API path the Agent's API access and key (see *The API path*).
## The SDK path
`@prismshadow/penguin-core` is the TypeScript SDK this agent itself runs on. Use it to build your own AI apps:
- An **Agent** loads its state (prompts, tools, skills) from `<root>/<project_id>/agents/<agent_id>/`. Creating an Agent whose directory is empty initializes it with defaults. - A **Session** is one conversation of an Agent inside a **Workspace** directory. - `session.run()` executes one task and streams every step (thinking, text, tool calls) as OmniMessages.
### Project location
Create the app in the current workspace directory by default (the `CWD` value from your Environment section), as a self-contained project — do not place it under `<app_data_dir>` (PenguinHarness's app data root) or depend on any path outside the project folder. When creating the app's agent, the data root defaults **under the working directory (CWD)** too: point `createAgent({ root })` at a directory inside the project, resolved from the source file so it stays relative:
```ts const agent = await createAgent({ root: path.join(import.meta.dirname, "penguin_data") }); ```
With every reference relative to the project, the user can move or copy the folder anywhere and it still runs.
### Keys and the data root — check before you build
**The app's Penguin data root must live inside the CWD workspace — never `~/.penguin`.** Point `createAgent({ root })` and every `penguin config ... --root <dir>` at a directory under the current working directory (e.g. `./penguin_data`); the global `~/.penguin` belongs to the person running Penguin and must never hold — or lend — the app's config or keys.
**Credential first, code second** — a finished app that cannot answer is a failed delivery discovered too late. Before writing any code:
```bash env | grep -oE "(DEEPSEEK|OPENAI|ANTHROPIC|GEMINI)_API_KEY" || echo none ```
**Only two sources count as a usable credential**: a vault-injected environment variable (the check above; vault keys also appear in your Vault Keys section), or a key already configured in the app's own data root (`penguin config model list --root <data_dir>`). Keys in the global `~/.penguin` or any other `.penguin` directory do **not** count — a bare `penguin config model list` (no `--root`) reads the global store, because the CLI defaults to the global root unless `--root` is given, so a key showing up there proves nothing for the app and must never be used or copied.
If neither counted source yields a key, **stop immediately and ask the user to configure one — do not start building, and do not burn turns re-checking in a loop**: have them open this agent's settings via the **gear icon** on its card (left side, Agents page) and add a model API key (e.g. `DEEPSEEK_API_KEY`) in the **key vault** tab — vault values reach your shell environment on the next task. One clear check, then hand back to the user. Build only after a credential is confirmed, or after clearly agreeing with the user to build now and verify later. Model ids to offer the user come from the penguin CLI catalog (`penguin config model add --help`) and the unified-llm-api skill's id table.
### Setup
```bash npm install @prismshadow/penguin-core tsx ```
If the package is not on your npm registry (it is developed in the PenguinHarness monorepo and may not be published), develop inside a checkout of the PenguinHarness repo instead: add your app as a workspace package under `packages/`, depend on `"@prismshadow/penguin-core": "workspace:*"`, then `pnpm install && pnpm build` at the repo root. Tell the user which route you took.
Configure a model for the app's data root, in this order — stop at the first that works:
1. `penguin config model add --root <data_dir> --provider <group> --model-id <id> --api-key <key> [--base-url <url>] [--client-type openai-chat] --set-default` — prefer `--client-type openai-chat --base-url <endpoint>` (works with any OpenAI Chat Completions compatible endpoint; exact ids in the unified-llm-api skill). `--provider` is required: a model is always the `(provider, model_id)` pair and the group is never inferred from the id (`custom` for an endpoint outside the built-in groups). 2. Environment variables cover the **credential only** (`DEEPSEEK_API_KEY`, `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, …), and only for a model with **no base URL** on itself or its group, or the vendor's own default endpoint — a model with any other base URL (a gateway, a self-hosted server) needs `--api-key`, or `createSession` refuses it. The SDK carries no model catalog: a data root only the SDK and the CLI touch starts with **no models and no default**, so model selection always comes from what step 1 or your code writes. For env-only setup, add the model in code before `createAgent` — `addModel(root, "default_project", { provider: "deepseek", model_id: "deepseek-flash" }, { setDefault: true })`, exported by `@prismshadow/penguin-core` — and leave its group's base URL empty.
Keep model API keys **project-local**: configure them with the penguin CLI into the app's own data root under the working directory, so the project stays self-contained and movable. When building an AI app, **always pass `--root <data_dir>` pointing at the app's data directory inside the current working directory** (the same path you give `createAgent({ root })`, e.g. `./penguin_data`) — never run `penguin config ...` without `--root`, or it writes to the global `~/.penguin/data` instead of the project. Never read, copy or fall back to model keys stored in the user's global `~/.penguin` directory — that config belongs to the person running Penguin, not to the app you are building.
Model config lives in one hidden file under the data root's project directory: `.project_config.toml`. It is CLI-only — never read, print or edit it.
If the user agreed to build before a credential exists, do not fake the verification: finish the build, report it as **unverified**, and point them at the key vault flow above — once a key is added, vault values reach your environment on the next task and you can run the self-test to completion.
### Streaming loop
The raw `run()` stream mixes model, event and session-meta payloads — always narrow with the exported guards (`isModelMessage`, `isCompleteModelMessage`, `isEventMessage`) before touching `payload.type`; accessing `msg.payload.type` directly does not typecheck.
```ts import path from "node:path"; import readline from "node:readline/promises"; import { createAgent, isModelMessage, userText } from "@prismshadow/penguin-core";
const agent = await createAgent({ root: path.join(import.meta.dirname, "penguin_data") }); const session = await agent.createSession({ workspaceDir: process.cwd() });
const rl = readline.createInterface({ input: process.stdin, output: process.stdout }); for (;;) { const line = await rl.question("> "); if (!line.trim()) break; // One run per user turn; the same Session keeps the conversation context. for await (const msg of session.run([userText(line)], { approve: async () => "allow", // demo only — a real app should ask its user ("deny" blocks the call) })) { if (isModelMessage(msg)) { const p = msg.payload; if (p.type === "partial_text" && p.event_type === "delta") process.stdout.write(p.text); // CoT stream from reasoning models — show progress, but keep it out of the answer channel. if (p.type === "partial_thinking" && p.event_type === "delta") process.stderr.write(p.thinking); } } process.stdout.write("\n"); } rl.close(); session.dispose(); ```
- `createSession({ workspaceDir, provider, modelId })` — `workspaceDir` must already exist (omit for a temporary workspace); the model reference is the `(provider, modelId)` pair, so pass both to pick a configured model or neither for the project default — passing one alone throws. - The `approve` callback gates every tool call; **omitting it denies everything**. - `opts.thinkingLevel` (`"none" | "low" | "medium" | "high" | "xhigh" | "max"`) overrides the agent's default (`model.thinking_level` in `system_config.yaml`) for this turn only — raise it for hard questions, drop it for latency-sensitive calls like titling or classification. - Session lifetime is the app's memory model: reuse one Session for a stateful chat (context accumulates, as above), create one per request for stateless QA (the RAG recipe below); either way call `session.dispose()` when done to release background processes. - An Agent's behavior is edited in its `agent_state/` files (system_config.yaml, AGENTS.md, skills/), not in code.
### Thinking and image messages
Modern models think before answering and accept images; the stream and the input protocol carry both — use them instead of flattening everything to text.
**Thinking (CoT) out.** Reasoning models stream `partial_thinking` (field `thinking`) before any `partial_text`, and a complete `thinking` message follows. Show the stream — a silent 20-second wait reads as a hang — but keep it in its own channel: a collapsible muted block per the web-design skill, auto-collapsed once answer text starts. Never concatenate thinking into the answer, store it as the answer, or cite from it; ignore its `fidelity` field (core's replay bookkeeping). Non-reasoning models simply never emit it — don't reserve UI space.
**Images in.** Build image input with `imageUrlMessage` (a web URL or a base64 data URL) beside `userText` in the same `run` input:
```ts import { imageUrlMessage, userText } from "@prismshadow/penguin-core"; session.run([userText(question), ...images.map(imageUrlMessage)], { ... }); ```
Browser flow: `<input type="file" accept="image/*">` plus paste/drag-drop → `FileReader.readAsDataURL` → POST `{ question, images: [dataUrl] }` → the server maps each entry to `imageUrlMessage`. Reject non-image MIME types and cap size (a data URL rides the context window; a few MB is plenty). Whether the session model actually sees pixels is the model config's `vision` flag (`penguin config model list` prints `vision=Y/-`; set via `--vision/--no-vision` on `model add`, default supported): with `vision=false` the core folds the image into an `[attached image: <path>]` line and the built-in `read_file` tool reads it through the project's configured `vision_model` (`penguin config model vision --provider <group> --model-id <id> --root <data_dir>`) — the app still works, through a description instead of direct sight.
**Other payloads worth handling** (always narrow with the guards first): `partial_tool_call` / `partial_tool_call_output` — surface as an activity line ("running `search`…") in apps that grant tools; `request_end` (event) — a non-`completed` `status` is the error signal (`auth` → ask for a key; `message` carries the failure detail; `retry_in_ms` announces a planned in-run retry, renderable as a countdown); `token_usage` (event) — session-cumulative and last-request counts, if the app shows cost; `compaction_begin` / `compaction_end` (events) — long-lived chats only, show a brief "context being compacted" notice. Everything else is safe to ignore.
### Wiring in the user's tools
When the app's agent must call the user's existing tools (scripts, internal CLIs, anything with an entry point), integrate them as **CLI commands** first: wrap each one as a small executable inside the project (a script under `tools/`, or the user's own binary), and describe it in the embedded agent's persona / `AGENTS.md` — name, what it does, one usage line. The agent invokes it through the built-in `exec_command` tool, so there is nothing to register: no schema to declare, arguments are flags, stdout is the result, the `approve` callback still gates every invocation, and the same command stays testable by hand.
Add an MCP server (`tools.mcpServers` in `system_config.yaml`) only when a CLI wrapper cannot express the integration — a long-lived authenticated connection, or tool schemas the model must see typed. Otherwise the CLI form is the cheaper default and keeps the project self-contained.
### RAG knowledge app
The default recipe when the user wants an app that answers questions over a document set ("docs QA", "knowledge base", "chat with our docs", "become an expert on X"). The core contributes the agent loop only — retrieval is app code. Default to **lexical BM25**: no extra dependencies, no embedding credential, works offline. (Semantic upgrade: embed chunks via `@prismshadow/mmsp` — see the unified-llm-api skill — and rank by cosine; only when an embedding-capable key is configured.)
``` my-app/ package.json # "type": "module"; scripts: ingest / start persona.md # the embedded agent's role — write it per the agent-initialization skill ingest.ts # corpus/ → data/index.json; initializes penguin_data/, installs persona rag.ts # BM25 retrieval over the chunk index server.ts # POST /api/ask streams SSE; serves public/ public/index.html # chat UI — build it per the web-design skill corpus/ # collected source documents data/index.json # generated chunk index penguin_data/ # agent data root (generated; model config lives here) ```
**Collect** — clone or fetch the sources into `corpus/`, keeping only text formats:
```bash git clone --depth 1 <repo_url> corpus/<name> # or curl pages into corpus/ find corpus -type f ! -regex '.*\.\(md\|mdx\|txt\|html?\)$' -delete && rm -rf corpus/*/.git ```
**Ingest** (`ingest.ts`) — split on markdown headings, cap chunk size, write one JSON index; also initialize `penguin_data/`, install the persona and strip the skills the embedded agent doesn't need:
```ts import fs from "node:fs"; import path from "node:path"; import { createAgent } from "@prismshadow/penguin-core";
const ROOT = import.meta.dirname; const walk = (d: string): string[] => fs.readdirSync(d, { withFileTypes: true }).flatMap((e) => e.isDirectory() ? walk(path.join(d, e.name)) : [path.join(d, e.name)]);
const STATE = path.join( ROOT, "penguin_data", "default_project", "agents", "default_agent", "agent_state"); await createAgent({ root: path.join(ROOT, "penguin_data") }); fs.copyFileSync(path.join(ROOT, "persona.md"), path.join(STATE, "AGENTS.md")); // A fresh default_agent is initialized with the whole built-in Skill library, and every installed // Skill's metadata is injected into the system prompt of every /api/ask. This app only answers // from retrieved context, so remove them: unrelated skill descriptions cost tokens on each // question and pull the answer off-topic when one happens to match the wording of a question. fs.rmSync(path.join(STATE, "skills"), { recursive: true, force: true });
const chunks: { id: number; source: string; heading: string; text: string }[] = []; for (const f of walk(path.join(ROOT, "corpus")).filter((f) => /\.(md|mdx|txt|html?)$/i.test(f))) { const raw = fs.readFileSync(f, "utf8"); const text = /\.html?$/i.test(f) ? raw.replace(/<[^>]+>/g, " ") : raw; const source = path.relative(ROOT, f); let heading = path.basename(f); for (const block of text.split(/^(?=#{1,3} )/m)) { heading = block.match(/^#{1,3} (.+)/)?.[1] ?? heading; for (let i = 0; i < block.length; i += 1500) { const piece = block.slice(i, i + 1500).trim(); if (piece.length > 40) chunks.push({ id: chunks.length, source, heading, text: piece }); } } } fs.mkdirSync(path.join(ROOT, "data"), { recursive: true }); fs.writeFileSync(path.join(ROOT, "data", "index.json"), JSON.stringify(chunks)); console.log(`indexed ${chunks.length} chunks`); ```
**Retrieve** (`rag.ts`) — standard BM25 (k1 = 1.2, b = 0.75); the tokenizer treats each CJK character as a token so Chinese queries work. The corpus-wide statistics (per-chunk term frequencies, document frequencies, average length) never change once the corpus is indexed, so build them **once** in `loadIndex` — a per-query rescan would make every question O(corpus):
```ts import fs from "node:fs"; import path from "node:path";
export interface Chunk { id: number; source: string; heading: string; text: string } export interface Index { chunks: Chunk[]; tf: Map<string, number>[]; // per-chunk term → count len: number[]; // per-chunk token length df: Map<string, number>; // term → number of chunks containing it avg: number; // mean chunk length (BM25 length normalization) }
const tokenize = (s: string): string[] => s.toLowerCase().match(/[a-z0-9]+|[一-鿿]/g) ?? [];
export function loadIndex(): Index { const chunks: Chunk[] = JSON.parse( fs.readFileSync(path.join(import.meta.dirname, "data", "index.json"), "utf8")); const tf: Map<string, number>[] = []; const len: number[] = []; const df = new Map<string, number>(); for (const c of chunks) { const toks = tokenize(`${c.heading} ${c.text}`); const m = new Map<string, number>(); for (const t of toks) m.set(t, (m.get(t) ?? 0) + 1); for (const t of m.keys()) df.set(t, (df.get(t) ?? 0) + 1); tf.push(m); len.push(toks.length); } const avg = len.reduce((n, l) => n + l, 0) / Math.max(len.length, 1); return { chunks, tf, len, df, avg }; }
export function search(index: Index, query: string, k = 6): Chunk[] { const { chunks, tf, len, df, avg } = index; const q = [...new Set(tokenize(query))]; const score = (i: number): number => { let s = 0; for (const t of q) { const f = tf[i]!.get(t) ?? 0; if (f === 0) continue; const n = df.get(t) ?? 0; s += Math.log(1 + (chunks.length - n + 0.5) / (n + 0.5)) * (f * 2.2) / (f + 1.2 * (0.25 + (0.75 * len[i]!) / avg)); } return s; }; return chunks.map((_, i) => [score(i), i] as const) .filter(([s]) => s > 0).sort((a, b) => b[0] - a[0]).slice(0, k) .map(([, i]) => chunks[i]!); } ```
**Answer & serve** (`server.ts`) — one Session per request (stateless QA), retrieved chunks numbered into the prompt, deltas streamed over SSE, sources sent as the final event. A pure QA session needs no tool calls — deny every approval; a denied or tool-less turn terminates normally. Do **not** clear the toolset with `tools: { builtin: [] }`: an empty tools array is sent to the provider verbatim and some OpenAI-compatible endpoints reject it with a 400, which surfaces as a silent empty answer. Guard the request boundary — a malformed body must return 400, never reject the async handler (an unhandled rejection takes the whole server down) — and abort the run if the client disconnects mid-answer so you stop generating (and paying) for a page nobody is reading.
```ts import fs from "node:fs"; import http from "node:http"; import path from "node:path"; import { createAgent, isModelMessage, userText } from "@prismshadow/penguin-core"; import { loadIndex, search } from "./rag.ts";
const ROOT = import.meta.dirname; const PUB = path.join(ROOT, "public"); const agent = await createAgent({ root: path.join(ROOT, "penguin_data") }); const index = loadIndex(); const MIME: Record<string, string> = { ".html": "text/html", ".css": "text/css", ".js": "text/javascript" };
http.createServer(async (req, res) => { res.on("error", () => {}); // a client that vanishes mid-write must not throw an uncaught EPIPE if (req.method === "POST" && req.url === "/api/ask") { let question: string; try { let body = ""; for await (const part of req) body += part; // a mid-body connection reset rejects here — caught below, never fatal const parsed = JSON.parse(body) as { question?: unknown }; if (typeof parsed.question !== "string" || !parsed.question.trim()) throw new Error(); question = parsed.question; } catch { res.writeHead(400, { "content-type": "application/json" }); res.end(JSON.stringify({ error: "expected a JSON body { question: string }" })); return; } const hits = search(index, question); const context = hits.map((c, i) => `[${i + 1}] ${c.source} — ${c.heading}\n${c.text}`).join("\n\n"); const ac = new AbortController(); res.on("close", () => ac.abort()); // client navigated away → cancel the in-flight generation // Create the Session BEFORE committing headers: a model-config failure then returns a real // HTTP error instead of an unhandled rejection with a 200 already on the wire. let session; try { session = await agent.createSession({ workspaceDir: ROOT }); } catch { res.writeHead(503, { "content-type": "application/json" }); res.end(JSON.stringify({ error: "no model configured yet — see the setup steps" })); return; } res.writeHead(200, { "content-type": "text/event-stream", "cache-control": "no-cache" }); try { const prompt = `Answer in plain text (no Markdown; short paragraphs) from the context below; cite blocks inline as [1][2]. If the context is not enough, say so.\n\n${context}\n\nQuestion: ${question}`; for await (const msg of session.run([userText(prompt)], { approve: async () => "deny", signal: ac.signal })) { if (isModelMessage(msg)) { const p = msg.payload; if (p.type === "partial_text" && p.event_type === "delta" && !res.writableEnded) res.write(`data: ${JSON.stringify({ delta: p.text })}\n\n`); // Reasoning models: forward CoT on its own SSE field so the UI can collapse it. if (p.type === "partial_thinking" && p.event_type === "delta" && !res.writableEnded) res.write(`data: ${JSON.stringify({ thinking: p.thinking })}\n\n`); } } // Sources carry the matched chunk text verbatim: the UI must be able to show the exact // block behind each [n], not just a file link. if (!res.writableEnded) res.write(`data: ${JSON.stringify({ sources: hits.map((c) => ({ source: c.source, heading: c.heading, url: `/${c.source}`, text: c.text })) })}\n\n`); } catch { // The run failed after headers were sent, or the client left: surface an error event (best effort), then clean up. if (!res.writableEnded) res.write(`data: ${JSON.stringify({ error: "generation failed" })}\n\n`); } finally { session.dispose(); if (!res.writableEnded) res.end(); } return; } const pathname = (req.url ?? "/").split("?")[0] ?? "/"; // /corpus/* serves the source documents read-only, so citation links resolve to real files. const inCorpus = pathname.startsWith("/corpus/"); const base = inCorpus ? path.join(ROOT, "corpus") : PUB; const rel = inCorpus ? pathname.slice("/corpus/".length) : pathname === "/" ? "index.html" : pathname.slice(1); const file = path.normalize(path.join(base, rel)); if (file.startsWith(base + path.sep) && fs.existsSync(file) && fs.statSync(file).isFile()) { res.writeHead(200, { "content-type": MIME[path.extname(file)] ?? "text/plain" }); res.end(fs.readFileSync(file)); } else { res.writeHead(404); res.end(); } }).listen(Number(process.env.PORT ?? 4630), () => console.log("http://localhost:4630")); ```
**UI** (`public/index.html`) — a chat interface built per the web-design skill: message list, streamed assistant text appended delta by delta (plain text under the output contract below: escape, split blank-line paragraphs, style the `[n]` markers), `thinking` events into the collapsible reasoning block (collapse it when the first answer delta arrives), the final `sources` event rendered as citations (pill chips or accordion source cards), an empty state inviting the first question with **3–4 example questions the corpus can actually answer** (clicking one submits it), and a visible error state when `/api/ask` fails. Citations must satisfy both of these, never bare text:
- **Reveal the original chunk**: clicking a citation chip (or an inline `[n]`) opens a popover/panel showing the matched chunk's `text` from the sources event **verbatim** — the numbering maps 1:1 to the context blocks in the prompt, so `[n]` always reveals exactly the block the answer drew on. - **Link to the real document**: inside the popover, `<a href="<url>" target="_blank">` using the `url` field (`/corpus/<path>`, which this server serves) — clicking the chip itself opens the popover, the document link lives within it. When the corpus was cloned from a public repository, prefer mapping the path to the canonical upstream page instead (e.g. the GitHub blob URL derived from the clone URL).
**Output format and language** — settle both up front, in the persona and the retriever, not in the UI:
- **No Markdown pipeline — set the output format instead**: instruct the embedded agent (in `persona.md` and the per-request prompt) to answer in plain text — short paragraphs separated by blank lines, citations as bare `[n]`, no Markdown syntax. The UI then only escapes the text, splits paragraphs and styles the `[n]` markers; there is no renderer to build. When richer structure genuinely matters, have the model emit a small whitelisted HTML subset (`<p> <ul> <li> <strong> <code>`) and sanitize to exactly that whitelist before inserting — never inject unsanitized model output. - **Cross-language retrieval**: the corpus and the user often speak different languages (English docs, Chinese questions), and BM25 is purely lexical — a Chinese question scores zero against English chunks. At ingest time derive a small bilingual keyword map for the corpus's core vocabulary (10–20 domain terms, e.g. `权限 → permissions / allow / deny`, `钩子 → hooks`) and expand query tokens through it in `search()` before scoring; keep the per-character CJK tokenizer. The persona already pins the answer language to the question's language.
**Persona** (`persona.md`) — the embedded agent's role, written per the agent-initialization skill. Shape: one role sentence ("You are an expert on X; you answer strictly from the provided context blocks"), citation and refusal rules, plain-text output (no Markdown — the output contract above), answer language follows the question.
### Verify before you hand over
Never declare the app done without running it:
1. `npm install` succeeds (or the workspace route builds). 2. Model configured for `penguin_data` (CLI or env var; no usable key → see Setup: ask the user to add one to this agent's key vault, and report the app as unverified for now). 3. `npm run ingest` prints `indexed N chunks` with N > 0. 4. Start `npm start` in the background, then ask a real question: `curl -N -sS -X POST localhost:4630/api/ask -H 'content-type: application/json' -d '{"question":"<something the corpus answers>"}'` — expect streamed `data:` deltas ending in a `sources` event that carries `source`, `url` **and the matched chunk `text`** per hit. If nothing streams, the model call failed: re-check step 2 and the provider endpoint before touching the code. Then `curl` one of the returned source `url`s — it must return the document, not a 404 (citation links have to resolve). 5. Open the UI (or screenshot it) to confirm the layout renders.
Fix any failure and re-verify; when the app accepts image input, one verification question must include a real image. Report with backtick-wrapped relative paths (`server.ts`, `public/index.html`, …), how to start the app, and the assumptions you made.
## The API path
The program calls an Agent that lives in PenguinHarness. You write the program; the Project's owner opens the Agent to it. Read [`reference/agent-api.md`](reference/agent-api.md) before you write any code: it has the steps to hand the owner, the client code, curl, the errors and the checks. These rules hold throughout:
- **Never open an Agent yourself.** Whether an Agent is open to programs is the owner's decision, and so is everything around it: API access, keys, keyless access, the API approval mode and the server-wide switch. An Agent must not be able to expose itself. The server refuses `penguin agent api enable`, `disable`, `set`, `keys create`, `keys rm` and `server` to the token in your shell with `403 human_required`, because they take a person's sign-in. Never run them, and never sign in or mint a session to get past that refusal. Ask the user to make the change on the Agent's **API** tab, or with those commands in their own terminal after `penguin auth login`, and give them the steps. `penguin agent api status` and `keys ls` only read; use them to check. - **Keep the key out of the code.** The program reads it from the `PENGUIN_AGENT_KEY` environment variable. Never write a key into source, a committed file, a log or a reply, and never ask the user to paste one into the chat. - **Connection values.** The Base URL and the Agent ID come from the API tab or `penguin agent api status --agent-id <agentId>`. A local server's Base URL is `http://localhost:7364/api/amsp/v1`: `localhost`, not `127.0.0.1`, where a local server serves previews and refuses every API request with `401`. The Agent ID is `<projectId>/<agentId>`. - **No data root, no model key.** The server runs the Agent on the Project's models. An SDK recipe keeps its shape on this path: the RAG app, for instance, replaces `createAgent`, `createSession` and `session.run` with an `AgentClient` run, and its persona goes into the Agent's `AGENTS.md`.
Intended uses
- Use Penguin Sdk when this documented workflow matches the task.
Implementation notes
- Review the source instructions and adapt inputs before running the workflow.
Try it, then tell us how it went
A copied prompt does not tell us whether the task worked. Report your result after checking the output. We collect your tool and outcome, never your files or prompt.