Skip to main content
Agent skill

Healthcare Eval Harness

Patient safety evaluation harness for healthcare application deployments. Automated test suites for CDSS accuracy, PHI exposure, clinical workflow integrity, and integration compliance. Blocks deployments on safety failures. Use when a healthcare deployment must be gated on patient-safety tests for CDSS accuracy, PHI exposure, and workflow integrity.

Get started

Get the complete skill folder, including its scripts and reference files.

·

When to use this skill

Use Healthcare Eval Harness when an AI agent needs a reusable SKILL.md workflow for this job: Patient safety evaluation harness for healthcare application deployments. Automated test suites for CDSS accuracy, PHI exposure, clinical workflow integrity, and integration compliance. Blocks deployments on safety failures. Use when a healthcare deployment must be gated on patient-safety tests for CDSS accuracy, PHI exposure, and workflow integrity.

When not to use it

Skip Healthcare Eval Harness when the task is outside the coding category, or when a more specific skill in this directory already covers the same workflow with clearer triggers.

How to install

Where will you use it?

Install scope
Claude Code folder setup
mkdir -p ~/.claude/skills/healthcare-eval-harness
# Save SKILL.md to ~/.claude/skills/healthcare-eval-harness/SKILL.md
  1. Claude Code personal install: place the skill and its bundled files at ~/.claude/skills/healthcare-eval-harness/SKILL.md. For a shared project, commit the folder at .claude/skills/healthcare-eval-harness/ instead.
  2. Current Claude Code detects SKILL.md changes in watched directories during the session. If you just created a top-level skills directory, run /reload-skills. Check that the skill is listed before testing its trigger.
  3. For Claude chat or Cowork in the Desktop app, manage uploaded skills in Customize > Skills; copying files into a local Claude Code folder is not a Desktop install. Other agents may use different install locations.

Full install guide for Claude, Cursor, and Codex →

Try a starter prompt

After installing the skill, paste this starter prompt into your agent to verify it triggers on a small task.

Starter prompt
Use the Healthcare Eval Harness skill for this task: Before any deployment of EMR/EHR applications. Confirm any required inputs or files before making changes.

What supports this recommendation?

83/100 provisional content score. This measures documentation, practical detail and source signals. It is not a measured task success rate or a security certification.

Documentation
92/100
Practical detail
92/100
Source evidence
63/100
Source signals
72/100
Model compatibility evidence
  • claudeuntested

    No model-specific signal or recorded compatibility test was found.

  • chatgptuntested

    No model-specific signal or recorded compatibility test was found.

  • geminiuntested

    No model-specific signal or recorded compatibility test was found.

  • copilotuntested

    No model-specific signal or recorded compatibility test was found.

  • llamauntested

    No model-specific signal or recorded compatibility test was found.

  • perplexityuntested

    No model-specific signal or recorded compatibility test was found.

  • mistraluntested

    No model-specific signal or recorded compatibility test was found.

  • grokuntested

    No model-specific signal or recorded compatibility test was found.

Inferred compatibility is based on content signals. Check the app, its available tools and setup requirements before running a skill.

What this skill does

# Healthcare Eval Harness — Patient Safety Verification

Automated verification system for healthcare application deployments. A single CRITICAL failure blocks deployment. Patient safety is non-negotiable.

> **Note:** Examples use Jest as the reference test runner. Adapt commands for your framework (Vitest, pytest, PHPUnit, etc.) — the test categories and pass thresholds are framework-agnostic.

## When to Use

- Before any deployment of EMR/EHR applications - After modifying CDSS logic (drug interactions, dose validation, scoring) - After changing database schemas that touch patient data - After modifying authentication or access control - During CI/CD pipeline configuration for healthcare apps - After resolving merge conflicts in clinical modules

## How It Works

The eval harness runs five test categories in order. The first three (CDSS Accuracy, PHI Exposure, Data Integrity) are CRITICAL gates requiring 100% pass rate — a single failure blocks deployment. The remaining two (Clinical Workflow, Integration) are HIGH gates requiring 95%+ pass rate.

Each category maps to a Jest test path pattern. The CI pipeline runs CRITICAL gates with `--bail` (stop on first failure) and enforces coverage thresholds with `--coverage --coverageThreshold`.

### Eval Categories

**1. CDSS Accuracy (CRITICAL — 100% required)**

Tests all clinical decision support logic: drug interaction pairs (both directions), dose validation rules, clinical scoring vs published specs, no false negatives, no silent failures.

```bash npx jest --testPathPattern='tests/cdss' --bail --ci --coverage ```

**2. PHI Exposure (CRITICAL — 100% required)**

Tests for protected health information leaks: API error responses, console output, URL parameters, browser storage, cross-facility isolation, unauthenticated access, service role key absence.

```bash npx jest --testPathPattern='tests/security/phi' --bail --ci ```

**3. Data Integrity (CRITICAL — 100% required)**

Tests clinical data safety: locked encounters, audit trail entries, cascade delete protection, concurrent edit handling, no orphaned records.

```bash npx jest --testPathPattern='tests/data-integrity' --bail --ci ```

**4. Clinical Workflow (HIGH — 95%+ required)**

Tests end-to-end flows: encounter lifecycle, template rendering, medication sets, drug/diagnosis search, prescription PDF, red flag alerts.

```bash tmp_json=$(mktemp) npx jest --testPathPattern='tests/clinical' --ci --json --outputFile="$tmp_json" || true total=$(jq '.numTotalTests // 0' "$tmp_json") passed=$(jq '.numPassedTests // 0' "$tmp_json") if [ "$total" -eq 0 ]; then echo "No clinical tests found" >&2 exit 1 fi rate=$(echo "scale=2; $passed * 100 / $total" | bc) echo "Clinical pass rate: ${rate}% ($passed/$total)" ```

**5. Integration Compliance (HIGH — 95%+ required)**

Tests external systems: HL7 message parsing (v2.x), FHIR validation, lab result mapping, malformed message handling.

```bash tmp_json=$(mktemp) npx jest --testPathPattern='tests/integration' --ci --json --outputFile="$tmp_json" || true total=$(jq '.numTotalTests // 0' "$tmp_json") passed=$(jq '.numPassedTests // 0' "$tmp_json") if [ "$total" -eq 0 ]; then echo "No integration tests found" >&2 exit 1 fi rate=$(echo "scale=2; $passed * 100 / $total" | bc) echo "Integration pass rate: ${rate}% ($passed/$total)" ```

### Pass/Fail Matrix

| Category | Threshold | On Failure | |----------|-----------|------------| | CDSS Accuracy | 100% | **BLOCK deployment** | | PHI Exposure | 100% | **BLOCK deployment** | | Data Integrity | 100% | **BLOCK deployment** | | Clinical Workflow | 95%+ | WARN, allow with review | | Integration | 95%+ | WARN, allow with review |

### CI/CD Integration

```yaml name: Healthcare Safety Gate on: [push, pull_request]

jobs: safety-gate: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - uses: actions/setup-node@v4 with: node-version: '20' - run: npm ci

# CRITICAL gates — 100% required, bail on first failure - name: CDSS Accuracy run: npx jest --testPathPattern='tests/cdss' --bail --ci --coverage --coverageThreshold='{"global":{"branches":80,"functions":80,"lines":80}}'

- name: PHI Exposure Check run: npx jest --testPathPattern='tests/security/phi' --bail --ci

- name: Data Integrity run: npx jest --testPathPattern='tests/data-integrity' --bail --ci

# HIGH gates — 95%+ required, custom threshold check # HIGH gates — 95%+ required - name: Clinical Workflows run: | TMP_JSON=$(mktemp) npx jest --testPathPattern='tests/clinical' --ci --json --outputFile="$TMP_JSON" || true TOTAL=$(jq '.numTotalTests // 0' "$TMP_JSON") PASSED=$(jq '.numPassedTests // 0' "$TMP_JSON") if [ "$TOTAL" -eq 0 ]; then echo "::error::No clinical tests found"; exit 1 fi RATE=$(echo "scale=2; $PASSED * 100 / $TOTAL" | bc) echo "Pass rate: ${RATE}% ($PASSED/$TOTAL)" if (( $(echo "$RATE < 95" | bc -l) )); then echo "::warning::Clinical pass rate ${RATE}% below 95%" fi

- name: Integration Compliance run: | TMP_JSON=$(mktemp) npx jest --testPathPattern='tests/integration' --ci --json --outputFile="$TMP_JSON" || true TOTAL=$(jq '.numTotalTests // 0' "$TMP_JSON") PASSED=$(jq '.numPassedTests // 0' "$TMP_JSON") if [ "$TOTAL" -eq 0 ]; then echo "::error::No integration tests found"; exit 1 fi RATE=$(echo "scale=2; $PASSED * 100 / $TOTAL" | bc) echo "Pass rate: ${RATE}% ($PASSED/$TOTAL)" if (( $(echo "$RATE < 95" | bc -l) )); then echo "::warning::Integration pass rate ${RATE}% below 95%" fi ```

### Anti-Patterns

- Skipping CDSS tests "because they passed last time" - Setting CRITICAL thresholds below 100% - Using `--no-bail` on CRITICAL test suites - Mocking the CDSS engine in integration tests (must test real logic) - Allowing deployments when safety gate is red - Running tests without `--coverage` on CDSS suites

## Examples

### Example 1: Run All Critical Gates Locally

```bash npx jest --testPathPattern='tests/cdss' --bail --ci --coverage && \ npx jest --testPathPattern='tests/security/phi' --bail --ci && \ npx jest --testPathPattern='tests/data-integrity' --bail --ci ```

### Example 2: Check HIGH Gate Pass Rate

```bash tmp_json=$(mktemp) npx jest --testPathPattern='tests/clinical' --ci --json --outputFile="$tmp_json" || true jq '{ passed: (.numPassedTests // 0), total: (.numTotalTests // 0), rate: (if (.numTotalTests // 0) == 0 then 0 else ((.numPassedTests // 0) / (.numTotalTests // 1) * 100) end) }' "$tmp_json" # Expected: { "passed": 21, "total": 22, "rate": 95.45 } ```

### Example 3: Eval Report

``` ## Healthcare Eval: 2026-03-27 [commit abc1234]

### Patient Safety: PASS

| Category | Tests | Pass | Fail | Status | |----------|-------|------|------|--------| | CDSS Accuracy | 39 | 39 | 0 | PASS | | PHI Exposure | 8 | 8 | 0 | PASS | | Data Integrity | 12 | 12 | 0 | PASS | | Clinical Workflow | 22 | 21 | 1 | 95.5% PASS | | Integration | 6 | 6 | 0 | PASS |

### Coverage: 84% (target: 80%+) ### Verdict: SAFE TO DEPLOY ```

Intended uses

  • Before any deployment of EMR/EHR applications
  • After modifying CDSS logic (drug interactions, dose validation, scoring)
  • After changing database schemas that touch patient data
  • After modifying authentication or access control
  • During CI/CD pipeline configuration for healthcare apps
  • After resolving merge conflicts in clinical modules

Implementation notes

  • Review the source instructions and adapt inputs before running the workflow.

Try it, then tell us how it went

A copied prompt does not tell us whether the task worked. Report your result after checking the output. We collect your tool and outcome, never your files or prompt.