Skip to main content
Agent skill

Analyze Regression

Compare two test262 result files to find exactly which tests changed status. Use after a test262 run shows unexpected pass count changes.

Get started

Get the complete skill folder, including its scripts and reference files.

When to use this skill

Use Analyze Regression when an AI agent needs a reusable SKILL.md workflow for this job: Compare two test262 result files to find exactly which tests changed status. Use after a test262 run shows unexpected pass count changes.

When not to use it

Skip Analyze Regression when the task is outside the coding category, or when a more specific skill in this directory already covers the same workflow with clearer triggers.

How to install

Where will you use it?

Install scope
Claude Code folder setup
mkdir -p ~/.claude/skills/analyze-regression
# Save SKILL.md to ~/.claude/skills/analyze-regression/SKILL.md
  1. Claude Code personal install: place the skill and its bundled files at ~/.claude/skills/analyze-regression/SKILL.md. For a shared project, commit the folder at .claude/skills/analyze-regression/ instead.
  2. Current Claude Code detects SKILL.md changes in watched directories during the session. If you just created a top-level skills directory, run /reload-skills. Check that the skill is listed before testing its trigger.
  3. For Claude chat or Cowork in the Desktop app, manage uploaded skills in Customize > Skills; copying files into a local Claude Code folder is not a Desktop install. Other agents may use different install locations.

Full install guide for Claude, Cursor, and Codex →

Try a starter prompt

After installing the skill, paste this starter prompt into your agent to verify it triggers on a small task.

Starter prompt
Use the Analyze Regression skill for this task: Use Analyze Regression when this documented workflow matches the task. Confirm any required inputs or files before making changes.

What supports this recommendation?

69/100 provisional content score. This measures documentation, practical detail and source signals. It is not a measured task success rate or a security certification.

Documentation
74/100
Practical detail
68/100
Source evidence
58/100
Source signals
72/100
Model compatibility evidence
  • claudeinferred

    The skill text mentions Claude or a closely associated term. This is keyword-derived compatibility; no hands-on model test is recorded.

  • chatgptuntested

    No model-specific signal or recorded compatibility test was found.

  • geminiuntested

    No model-specific signal or recorded compatibility test was found.

  • copilotuntested

    No model-specific signal or recorded compatibility test was found.

  • llamauntested

    No model-specific signal or recorded compatibility test was found.

  • perplexityuntested

    No model-specific signal or recorded compatibility test was found.

  • mistraluntested

    No model-specific signal or recorded compatibility test was found.

  • grokuntested

    No model-specific signal or recorded compatibility test was found.

Inferred compatibility is based on content signals. Check the app, its available tools and setup requirements before running a skill.

What this skill does

# Analyze Regression

Diffs two test262 JSONL result files to find exactly which tests flipped status.

## Steps

1. List available result archives: ```bash ls -lt /workspace/benchmarks/results/test262-results*.jsonl | head -10 ```

2. Pick the two files to compare (current vs previous): ```bash CURRENT="/workspace/benchmarks/results/test262-results.jsonl" PREVIOUS="/workspace/benchmarks/results/test262-results-YYYYMMDD-HHMMSS.jsonl" ```

3. Build status maps and diff: ```bash python3 -c " import json from collections import Counter

current, previous = {}, {} with open('$CURRENT') as f: for line in f: r = json.loads(line) current[r['file']] = r['status'] with open('$PREVIOUS') as f: for line in f: r = json.loads(line) previous[r['file']] = r['status']

# Find flips regressions = [] # was pass, now not fixes = [] # was not pass, now pass for test in set(current) | set(previous): old = previous.get(test, 'missing') new = current.get(test, 'missing') if old == 'pass' and new != 'pass': regressions.append((test, old, new)) elif old != 'pass' and new == 'pass': fixes.append((test, old, new))

print(f'Regressions (pass→fail): {len(regressions)}') for t, o, n in sorted(regressions)[:20]: print(f' {n:15s} ← {o:15s} {t}') if len(regressions) > 20: print(f' ... and {len(regressions)-20} more')

print(f'\nFixes (fail→pass): {len(fixes)}') for t, o, n in sorted(fixes)[:20]: print(f' {n:15s} ← {o:15s} {t}') if len(fixes) > 20: print(f' ... and {len(fixes)-20} more')

# Categorize regressions by new status regr_cats = Counter(n for _, _, n in regressions) print(f'\nRegression breakdown: {dict(regr_cats)}') " ```

4. Report findings: - Number of regressions and fixes - Top regression patterns (which status did they flip to?) - Common file paths (is it one codegen area?)

## Output

Message with: `"Regression analysis: +N fixes, -M regressions. Top pattern: [description]"`

Intended uses

  • Use Analyze Regression when this documented workflow matches the task.

Implementation notes

  • Review the source instructions and adapt inputs before running the workflow.

Try it, then tell us how it went

A copied prompt does not tell us whether the task worked. Report your result after checking the output. We collect your tool and outcome, never your files or prompt.