Regex Vs Llm Structured Text
Decision framework for parsing structured text (quizzes, forms, invoices, receipts, tables) with a hybrid regex-first pipeline — regex extraction handles 95%+ cheaply, a confidence scorer flags low-confidence items, and an LLM validator fixes only the edge cases. Use when choosing between regex and LLM for text extraction, building a cheap document parser, or optimizing extraction cost and accuracy.
Get the complete skill folder, including its scripts and reference files.
·
When to use this skill
Use Regex Vs Llm Structured Text when an AI agent needs a reusable SKILL.md workflow for this job: Decision framework for parsing structured text (quizzes, forms, invoices, receipts, tables) with a hybrid regex-first pipeline — regex extraction handles 95%+ cheaply, a confidence scorer flags low-confidence items, and an LLM validator fixes only the edge cases. Use when choosing between regex and LLM for text extraction, building a cheap document parser, or optimizing extraction cost and accuracy.
When not to use it
Skip Regex Vs Llm Structured Text when the task is outside the analysis category, or when a more specific skill in this directory already covers the same workflow with clearer triggers.
How to install
Where will you use it?
mkdir -p ~/.claude/skills/regex-vs-llm-structured-text # Save SKILL.md to ~/.claude/skills/regex-vs-llm-structured-text/SKILL.md
- Claude Code personal install: place the skill and its bundled files at ~/.claude/skills/regex-vs-llm-structured-text/SKILL.md. For a shared project, commit the folder at .claude/skills/regex-vs-llm-structured-text/ instead.
- Current Claude Code detects SKILL.md changes in watched directories during the session. If you just created a top-level skills directory, run /reload-skills. Check that the skill is listed before testing its trigger.
- For Claude chat or Cowork in the Desktop app, manage uploaded skills in Customize > Skills; copying files into a local Claude Code folder is not a Desktop install. Other agents may use different install locations.
Full install guide for Claude, Cursor, and Codex →
Try a starter prompt
After installing the skill, paste this starter prompt into your agent to verify it triggers on a small task.
Use the Regex Vs Llm Structured Text skill for this task: Quiz/exam question parsing. Confirm any required inputs or files before making changes.
What supports this recommendation?
88/100 provisional content score. This measures documentation, practical detail and source signals. It is not a measured task success rate or a security certification.
- Documentation
- 100/100
- Practical detail
- 100/100
- Source evidence
- 63/100
- Source signals
- 72/100
Model compatibility evidence
- claudeinferred
The skill text mentions Claude or a closely associated term. This is keyword-derived compatibility; no hands-on model test is recorded.
- chatgptinferred
The skill uses model-agnostic prompt or LLM terminology. Portability is inferred and has not been verified on this model.
- geminiinferred
The skill uses model-agnostic prompt or LLM terminology. Portability is inferred and has not been verified on this model.
- copilotinferred
The skill uses model-agnostic prompt or LLM terminology. Portability is inferred and has not been verified on this model.
- llamainferred
The skill uses model-agnostic prompt or LLM terminology. Portability is inferred and has not been verified on this model.
- perplexityinferred
The skill uses model-agnostic prompt or LLM terminology. Portability is inferred and has not been verified on this model.
- mistralinferred
The skill uses model-agnostic prompt or LLM terminology. Portability is inferred and has not been verified on this model.
- grokinferred
The skill uses model-agnostic prompt or LLM terminology. Portability is inferred and has not been verified on this model.
Inferred compatibility is based on content signals. Check the app, its available tools and setup requirements before running a skill.
What this skill does
# Regex vs LLM for Structured Text Parsing
A practical decision framework for parsing structured text (quizzes, forms, invoices, documents). The key insight: regex handles 95-98% of cases cheaply and deterministically. Reserve expensive LLM calls for the remaining edge cases.
## When to Activate
- Parsing structured text with repeating patterns (questions, forms, tables) - Deciding between regex and LLM for text extraction - Building hybrid pipelines that combine both approaches - Optimizing cost/accuracy tradeoffs in text processing
## Decision Framework
``` Is the text format consistent and repeating? ├── Yes (>90% follows a pattern) → Start with Regex │ ├── Regex handles 95%+ → Done, no LLM needed │ └── Regex handles <95% → Add LLM for edge cases only └── No (free-form, highly variable) → Use LLM directly ```
## Architecture Pattern
``` Source Text │ ▼ [Regex Parser] ─── Extracts structure (95-98% accuracy) │ ▼ [Text Cleaner] ─── Removes noise (markers, page numbers, artifacts) │ ▼ [Confidence Scorer] ─── Flags low-confidence extractions │ ├── High confidence (≥0.95) → Direct output │ └── Low confidence (<0.95) → [LLM Validator] → Output ```
## Implementation
### 1. Regex Parser (Handles the Majority)
```python import re from dataclasses import dataclass
@dataclass(frozen=True) class ParsedItem: id: str text: str choices: tuple[str, ...] answer: str confidence: float = 1.0
def parse_structured_text(content: str) -> list[ParsedItem]: """Parse structured text using regex patterns.""" pattern = re.compile( r"(?P<id>\d+)\.\s*(?P<text>.+?)\n" r"(?P<choices>(?:[A-D]\..+?\n)+)" r"Answer:\s*(?P<answer>[A-D])", re.MULTILINE | re.DOTALL, ) items = [] for match in pattern.finditer(content): choices = tuple( c.strip() for c in re.findall(r"[A-D]\.\s*(.+)", match.group("choices")) ) items.append(ParsedItem( id=match.group("id"), text=match.group("text").strip(), choices=choices, answer=match.group("answer"), )) return items ```
### 2. Confidence Scoring
Flag items that may need LLM review:
```python @dataclass(frozen=True) class ConfidenceFlag: item_id: str score: float reasons: tuple[str, ...]
def score_confidence(item: ParsedItem) -> ConfidenceFlag: """Score extraction confidence and flag issues.""" reasons = [] score = 1.0
if len(item.choices) < 3: reasons.append("few_choices") score -= 0.3
if not item.answer: reasons.append("missing_answer") score -= 0.5
if len(item.text) < 10: reasons.append("short_text") score -= 0.2
return ConfidenceFlag( item_id=item.id, score=max(0.0, score), reasons=tuple(reasons), )
def identify_low_confidence( items: list[ParsedItem], threshold: float = 0.95, ) -> list[ConfidenceFlag]: """Return items below confidence threshold.""" flags = [score_confidence(item) for item in items] return [f for f in flags if f.score < threshold] ```
### 3. LLM Validator (Edge Cases Only)
```python def validate_with_llm( item: ParsedItem, original_text: str, client, ) -> ParsedItem: """Use LLM to fix low-confidence extractions.""" response = client.messages.create( model="claude-haiku-4-5-20251001", # Cheapest model for validation max_tokens=500, messages=[{ "role": "user", "content": ( f"Extract the question, choices, and answer from this text.\n\n" f"Text: {original_text}\n\n" f"Current extraction: {item}\n\n" f"Return corrected JSON if needed, or 'CORRECT' if accurate." ), }], ) # Parse LLM response and return corrected item... return corrected_item ```
### 4. Hybrid Pipeline
```python def process_document( content: str, *, llm_client=None, confidence_threshold: float = 0.95, ) -> list[ParsedItem]: """Full pipeline: regex -> confidence check -> LLM for edge cases.""" # Step 1: Regex extraction (handles 95-98%) items = parse_structured_text(content)
# Step 2: Confidence scoring low_confidence = identify_low_confidence(items, confidence_threshold)
if not low_confidence or llm_client is None: return items
# Step 3: LLM validation (only for flagged items) low_conf_ids = {f.item_id for f in low_confidence} result = [] for item in items: if item.id in low_conf_ids: result.append(validate_with_llm(item, content, llm_client)) else: result.append(item)
return result ```
## Real-World Metrics
From a production quiz parsing pipeline (410 items):
| Metric | Value | |--------|-------| | Regex success rate | 98.0% | | Low confidence items | 8 (2.0%) | | LLM calls needed | ~5 | | Cost savings vs all-LLM | ~95% | | Test coverage | 93% |
## Best Practices
- **Start with regex** — even imperfect regex gives you a baseline to improve - **Use confidence scoring** to programmatically identify what needs LLM help - **Use the cheapest LLM** for validation (Haiku-class models are sufficient) - **Never mutate** parsed items — return new instances from cleaning/validation steps - **TDD works well** for parsers — write tests for known patterns first, then edge cases - **Log metrics** (regex success rate, LLM call count) to track pipeline health
## Anti-Patterns to Avoid
- Sending all text to an LLM when regex handles 95%+ of cases (expensive and slow) - Using regex for free-form, highly variable text (LLM is better here) - Skipping confidence scoring and hoping regex "just works" - Mutating parsed objects during cleaning/validation steps - Not testing edge cases (malformed input, missing fields, encoding issues)
## When to Use
- Quiz/exam question parsing - Form data extraction - Invoice/receipt processing - Document structure parsing (headers, sections, tables) - Any structured text with repeating patterns where cost matters
Intended uses
- Quiz/exam question parsing
- Form data extraction
- Invoice/receipt processing
- Document structure parsing (headers, sections, tables)
- Any structured text with repeating patterns where cost matters
Implementation notes
- Start with regex — even imperfect regex gives you a baseline to improve
- Use confidence scoring to programmatically identify what needs LLM help
- Use the cheapest LLM for validation (Haiku-class models are sufficient)
- Never mutate parsed items — return new instances from cleaning/validation steps
- TDD works well for parsers — write tests for known patterns first, then edge cases
- Log metrics (regex success rate, LLM call count) to track pipeline health
Try it, then tell us how it went
A copied prompt does not tell us whether the task worked. Report your result after checking the output. We collect your tool and outcome, never your files or prompt.