Skip to main content
Agent skill

Cost Aware Llm Pipeline

Cost optimization patterns for LLM API usage — model routing by task complexity, budget tracking, retry logic, and prompt caching. Use when LLM spend needs to come down, or when routing tasks across model tiers and budgets.

Get started

Get the complete skill folder, including its scripts and reference files.

·

When to use this skill

Use Cost Aware Llm Pipeline when an AI agent needs a reusable SKILL.md workflow for this job: Cost optimization patterns for LLM API usage — model routing by task complexity, budget tracking, retry logic, and prompt caching. Use when LLM spend needs to come down, or when routing tasks across model tiers and budgets.

When not to use it

Skip Cost Aware Llm Pipeline when the task is outside the coding category, or when a more specific skill in this directory already covers the same workflow with clearer triggers.

How to install

Where will you use it?

Install scope
Claude Code folder setup
mkdir -p ~/.claude/skills/cost-aware-llm-pipeline
# Save SKILL.md to ~/.claude/skills/cost-aware-llm-pipeline/SKILL.md
  1. Claude Code personal install: place the skill and its bundled files at ~/.claude/skills/cost-aware-llm-pipeline/SKILL.md. For a shared project, commit the folder at .claude/skills/cost-aware-llm-pipeline/ instead.
  2. Current Claude Code detects SKILL.md changes in watched directories during the session. If you just created a top-level skills directory, run /reload-skills. Check that the skill is listed before testing its trigger.
  3. For Claude chat or Cowork in the Desktop app, manage uploaded skills in Customize > Skills; copying files into a local Claude Code folder is not a Desktop install. Other agents may use different install locations.

Full install guide for Claude, Cursor, and Codex →

Try a starter prompt

After installing the skill, paste this starter prompt into your agent to verify it triggers on a small task.

Starter prompt
Use the Cost Aware Llm Pipeline skill for this task: Any application calling Claude, OpenAI, or similar LLM APIs. Confirm any required inputs or files before making changes.

What supports this recommendation?

88/100 provisional content score. This measures documentation, practical detail and source signals. It is not a measured task success rate or a security certification.

Documentation
100/100
Practical detail
100/100
Source evidence
63/100
Source signals
72/100
Model compatibility evidence
  • claudeinferred

    The skill text mentions Claude or a closely associated term. This is keyword-derived compatibility; no hands-on model test is recorded.

  • chatgptinferred

    The skill text mentions ChatGPT or a closely associated term. This is keyword-derived compatibility; no hands-on model test is recorded.

  • geminiinferred

    The skill uses model-agnostic prompt or LLM terminology. Portability is inferred and has not been verified on this model.

  • copilotinferred

    The skill uses model-agnostic prompt or LLM terminology. Portability is inferred and has not been verified on this model.

  • llamainferred

    The skill uses model-agnostic prompt or LLM terminology. Portability is inferred and has not been verified on this model.

  • perplexityinferred

    The skill uses model-agnostic prompt or LLM terminology. Portability is inferred and has not been verified on this model.

  • mistralinferred

    The skill uses model-agnostic prompt or LLM terminology. Portability is inferred and has not been verified on this model.

  • grokinferred

    The skill uses model-agnostic prompt or LLM terminology. Portability is inferred and has not been verified on this model.

Inferred compatibility is based on content signals. Check the app, its available tools and setup requirements before running a skill.

What this skill does

# Cost-Aware LLM Pipeline

Patterns for controlling LLM API costs while maintaining quality. Combines model routing, budget tracking, retry logic, and prompt caching into a composable pipeline.

## When to Activate

- Building applications that call LLM APIs (Claude, GPT, etc.) - Processing batches of items with varying complexity - Need to stay within a budget for API spend - Optimizing cost without sacrificing quality on complex tasks

## Core Concepts

### 1. Model Routing by Task Complexity

Automatically select cheaper models for simple tasks, reserving expensive models for complex ones.

```python MODEL_SONNET = "claude-sonnet-5" MODEL_HAIKU = "claude-haiku-4-5-20251001"

_SONNET_TEXT_THRESHOLD = 10_000 # chars _SONNET_ITEM_THRESHOLD = 30 # items

def select_model( text_length: int, item_count: int, force_model: str | None = None, ) -> str: """Select model based on task complexity.""" if force_model is not None: return force_model if text_length >= _SONNET_TEXT_THRESHOLD or item_count >= _SONNET_ITEM_THRESHOLD: return MODEL_SONNET # Complex task return MODEL_HAIKU # Simple task (3-4x cheaper) ```

### 2. Immutable Cost Tracking

Track cumulative spend with frozen dataclasses. Each API call returns a new tracker — never mutates state.

```python from dataclasses import dataclass

@dataclass(frozen=True, slots=True) class CostRecord: model: str input_tokens: int output_tokens: int cost_usd: float

@dataclass(frozen=True, slots=True) class CostTracker: budget_limit: float = 1.00 records: tuple[CostRecord, ...] = ()

def add(self, record: CostRecord) -> "CostTracker": """Return new tracker with added record (never mutates self).""" return CostTracker( budget_limit=self.budget_limit, records=(*self.records, record), )

@property def total_cost(self) -> float: return sum(r.cost_usd for r in self.records)

@property def over_budget(self) -> bool: return self.total_cost > self.budget_limit ```

### 3. Narrow Retry Logic

Retry only on transient errors. Fail fast on authentication or bad request errors.

```python from anthropic import ( APIConnectionError, InternalServerError, RateLimitError, )

_RETRYABLE_ERRORS = (APIConnectionError, RateLimitError, InternalServerError) _MAX_RETRIES = 3

def call_with_retry(func, *, max_retries: int = _MAX_RETRIES): """Retry only on transient errors, fail fast on others.""" for attempt in range(max_retries): try: return func() except _RETRYABLE_ERRORS: if attempt == max_retries - 1: raise time.sleep(2 ** attempt) # Exponential backoff # AuthenticationError, BadRequestError etc. → raise immediately ```

### 4. Prompt Caching

Cache long system prompts to avoid resending them on every request.

```python messages = [ { "role": "user", "content": [ { "type": "text", "text": system_prompt, "cache_control": {"type": "ephemeral"}, # Cache this }, { "type": "text", "text": user_input, # Variable part }, ], } ] ```

## Composition

Combine all four techniques in a single pipeline function:

```python def process(text: str, config: Config, tracker: CostTracker) -> tuple[Result, CostTracker]: # 1. Route model model = select_model(len(text), estimated_items, config.force_model)

# 2. Check budget if tracker.over_budget: raise BudgetExceededError(tracker.total_cost, tracker.budget_limit)

# 3. Call with retry + caching response = call_with_retry(lambda: client.messages.create( model=model, messages=build_cached_messages(system_prompt, text), ))

# 4. Track cost (immutable) record = CostRecord(model=model, input_tokens=..., output_tokens=..., cost_usd=...) tracker = tracker.add(record)

return parse_result(response), tracker ```

## Pricing Reference (2026)

| Model | Input ($/1M tokens) | Output ($/1M tokens) | Relative Cost | |-------|---------------------|----------------------|---------------| | Haiku 3.5 (legacy) | $0.80 | $4.00 | 0.8x | | Haiku 4.5 | $1.00 | $5.00 | 1x | | Sonnet 5 | $2.00 | $10.00 | 2x | | Sonnet 4.6 | $3.00 | $15.00 | 3x | | Opus 4.8 | $5.00 | $25.00 | 5x | | Fable 5 / Mythos 5 | $10.00 | $50.00 | 10x | | Opus 4.0 / 4.1 (legacy) | $15.00 | $75.00 | 15x |

## Best Practices

- **Start with the cheapest model** and only route to expensive models when complexity thresholds are met - **Set explicit budget limits** before processing batches — fail early rather than overspend - **Log model selection decisions** so you can tune thresholds based on real data - **Use prompt caching** for system prompts over 1024 tokens — saves both cost and latency - **Never retry on authentication or validation errors** — only transient failures (network, rate limit, server error)

## Anti-Patterns to Avoid

- Using the most expensive model for all requests regardless of complexity - Retrying on all errors (wastes budget on permanent failures) - Mutating cost tracking state (makes debugging and auditing difficult) - Hardcoding model names throughout the codebase (use constants or config) - Ignoring prompt caching for repetitive system prompts

## When to Use

- Any application calling Claude, OpenAI, or similar LLM APIs - Batch processing pipelines where cost adds up quickly - Multi-model architectures that need intelligent routing - Production systems that need budget guardrails

Intended uses

  • Any application calling Claude, OpenAI, or similar LLM APIs
  • Batch processing pipelines where cost adds up quickly
  • Multi-model architectures that need intelligent routing
  • Production systems that need budget guardrails

Implementation notes

  • Start with the cheapest model and only route to expensive models when complexity thresholds are met
  • Set explicit budget limits before processing batches — fail early rather than overspend
  • Log model selection decisions so you can tune thresholds based on real data
  • Use prompt caching for system prompts over 1024 tokens — saves both cost and latency
  • Never retry on authentication or validation errors — only transient failures (network, rate limit, server error)

Try it, then tell us how it went

A copied prompt does not tell us whether the task worked. Report your result after checking the output. We collect your tool and outcome, never your files or prompt.