Skip to main content
Agent skill

Django Celery

Django + Celery async task patterns — configuration, task design, beat scheduling, retries, canvas workflows, monitoring, and testing. Use when adding background jobs, scheduled tasks, or async processing to a Django app.

Get started

Get the complete skill folder, including its scripts and reference files.

·

When to use this skill

Use Django Celery when an AI agent needs a reusable SKILL.md workflow for this job: Django + Celery async task patterns — configuration, task design, beat scheduling, retries, canvas workflows, monitoring, and testing. Use when adding background jobs, scheduled tasks, or async processing to a Django app.

When not to use it

Skip Django Celery when the task is outside the coding category, or when a more specific skill in this directory already covers the same workflow with clearer triggers.

How to install

Where will you use it?

Install scope
Claude Code folder setup
mkdir -p ~/.claude/skills/django-celery
# Save SKILL.md to ~/.claude/skills/django-celery/SKILL.md
  1. Claude Code personal install: place the skill and its bundled files at ~/.claude/skills/django-celery/SKILL.md. For a shared project, commit the folder at .claude/skills/django-celery/ instead.
  2. Current Claude Code detects SKILL.md changes in watched directories during the session. If you just created a top-level skills directory, run /reload-skills. Check that the skill is listed before testing its trigger.
  3. For Claude chat or Cowork in the Desktop app, manage uploaded skills in Customize > Skills; copying files into a local Claude Code folder is not a Desktop install. Other agents may use different install locations.

Full install guide for Claude, Cursor, and Codex →

Try a starter prompt

After installing the skill, paste this starter prompt into your agent to verify it triggers on a small task.

Starter prompt
Use the Django Celery skill for this task: Use Django Celery when this documented workflow matches the task. Confirm any required inputs or files before making changes.

What supports this recommendation?

71/100 provisional content score. This measures documentation, practical detail and source signals. It is not a measured task success rate or a security certification.

Documentation
77/100
Practical detail
68/100
Source evidence
63/100
Source signals
72/100
Model compatibility evidence
  • claudeuntested

    No model-specific signal or recorded compatibility test was found.

  • chatgptuntested

    No model-specific signal or recorded compatibility test was found.

  • geminiuntested

    No model-specific signal or recorded compatibility test was found.

  • copilotuntested

    No model-specific signal or recorded compatibility test was found.

  • llamauntested

    No model-specific signal or recorded compatibility test was found.

  • perplexityuntested

    No model-specific signal or recorded compatibility test was found.

  • mistraluntested

    No model-specific signal or recorded compatibility test was found.

  • grokuntested

    No model-specific signal or recorded compatibility test was found.

Inferred compatibility is based on content signals. Check the app, its available tools and setup requirements before running a skill.

What this skill does

# Django + Celery Async Task Patterns

Production-grade patterns for background task processing in Django using Celery with Redis or RabbitMQ.

## When to Activate

- Adding background jobs or async processing to a Django app - Implementing periodic/scheduled tasks - Offloading slow operations (email, PDF generation, API calls) from request cycle - Setting up Celery Beat for cron-like scheduling - Debugging task failures, retries, or queue backlogs - Writing tests for Celery tasks

## Project Setup

### Installation

```bash pip install 'celery[redis]' django-celery-results django-celery-beat ```

### `celery.py` — App Entrypoint

```python # config/celery.py import os from celery import Celery

os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'config.settings.development')

app = Celery('myproject') app.config_from_object('django.conf:settings', namespace='CELERY') app.autodiscover_tasks() # Discovers tasks.py in each INSTALLED_APP

@app.task(bind=True, ignore_result=True) def debug_task(self): print(f'Request: {self.request!r}') ```

```python # config/__init__.py from .celery import app as celery_app

__all__ = ('celery_app',) ```

### Django Settings

```python # config/settings/base.py

# Broker (Redis recommended for production) CELERY_BROKER_URL = env('CELERY_BROKER_URL', default='redis://localhost:6379/0') CELERY_RESULT_BACKEND = env('CELERY_RESULT_BACKEND', default='django-db')

# Serialization CELERY_ACCEPT_CONTENT = ['json'] CELERY_TASK_SERIALIZER = 'json' CELERY_RESULT_SERIALIZER = 'json'

# Task behavior CELERY_TASK_TRACK_STARTED = True CELERY_TASK_TIME_LIMIT = 30 * 60 # Hard limit: 30 min CELERY_TASK_SOFT_TIME_LIMIT = 25 * 60 # Soft limit: sends SoftTimeLimitExceeded CELERY_WORKER_PREFETCH_MULTIPLIER = 1 # Prevent worker hoarding long tasks CELERY_TASK_ACKS_LATE = True # Re-queue on worker crash

# Result persistence CELERY_RESULT_EXPIRES = 60 * 60 * 24 # Keep results 24 hours

# Beat scheduler (for periodic tasks) CELERY_BEAT_SCHEDULER = 'django_celery_beat.schedulers:DatabaseScheduler'

# Installed apps INSTALLED_APPS += [ 'django_celery_results', 'django_celery_beat', ] ```

### Running Workers

```bash # Start worker (development) celery -A config worker --loglevel=info

# Start beat scheduler (periodic tasks) celery -A config beat --loglevel=info --scheduler django_celery_beat.schedulers:DatabaseScheduler

# Combined worker + beat (dev only, never production) celery -A config worker --beat --loglevel=info

# Production: multiple workers with concurrency celery -A config worker --loglevel=warning --concurrency=4 -Q default,high_priority ```

## Task Design Patterns

### Basic Task

```python # apps/notifications/tasks.py from celery import shared_task import logging

logger = logging.getLogger(__name__)

@shared_task(name='notifications.send_welcome_email') def send_welcome_email(user_id: int) -> None: """Send welcome email to newly registered user.""" from apps.users.models import User from apps.notifications.services import EmailService

try: user = User.objects.get(pk=user_id) except User.DoesNotExist: logger.warning('send_welcome_email: user %s not found', user_id) return # Idempotent — do not raise, task already impossible to complete

EmailService.send_welcome(user) logger.info('Welcome email sent to user %s', user_id) ```

### Retryable Task

```python @shared_task( bind=True, name='integrations.sync_to_crm', max_retries=5, default_retry_delay=60, # seconds before first retry autoretry_for=(ConnectionError, TimeoutError), retry_backoff=True, # exponential backoff retry_backoff_max=600, # cap at 10 minutes retry_jitter=True, # randomise to avoid thundering herd ) def sync_contact_to_crm(self, contact_id: int) -> dict: """Sync contact to external CRM with retry on transient failures.""" from apps.crm.services import CRMClient

try: result = CRMClient().sync(contact_id) return result except CRMClient.RateLimitError as exc: # Specific retry delay from response header raise self.retry(exc=exc, countdown=int(exc.retry_after)) ```

### Idempotent Task Pattern

Design tasks so they can safely run multiple times with the same inputs:

```python @shared_task(name='orders.mark_shipped') def mark_order_shipped(order_id: int, tracking_number: str) -> None: """Mark order as shipped — safe to run multiple times.""" from apps.orders.models import Order

updated = Order.objects.filter( pk=order_id, status=Order.Status.PROCESSING, # Guard: only update if not already shipped ).update( status=Order.Status.SHIPPED, tracking_number=tracking_number, )

if not updated: logger.info('mark_order_shipped: order %s already shipped or not found', order_id) ```

### Task with Soft Time Limit

```python from celery.exceptions import SoftTimeLimitExceeded

@shared_task( bind=True, name='reports.generate_pdf', soft_time_limit=120, time_limit=150, ) def generate_pdf_report(self, report_id: int) -> str: """Generate PDF report with graceful timeout handling.""" from apps.reports.services import PDFGenerator

try: path = PDFGenerator.build(report_id) return path except SoftTimeLimitExceeded: # Clean up partial files before hard kill PDFGenerator.cleanup(report_id) raise ```

## Calling Tasks

```python from datetime import timedelta from django.utils import timezone

# Fire and forget (async) send_welcome_email.delay(user.pk)

# Schedule in the future send_reminder.apply_async(args=[user.pk], countdown=3600) # 1 hour from now send_reminder.apply_async(args=[user.pk], eta=timezone.now() + timedelta(days=1))

# Apply with queue routing sync_contact_to_crm.apply_async(args=[contact.pk], queue='high_priority')

# Run synchronously (tests / debugging only) result = generate_pdf_report.apply(args=[report.pk]) ```

## Beat Scheduling (Periodic Tasks)

### Code-Defined Schedule

```python # config/settings/base.py from celery.schedules import crontab

CELERY_BEAT_SCHEDULE = { 'cleanup-expired-sessions': { 'task': 'users.cleanup_expired_sessions', 'schedule': crontab(hour=2, minute=0), # 2am daily }, 'sync-inventory': { 'task': 'products.sync_inventory', 'schedule': 60.0, # every 60 seconds }, 'weekly-digest': { 'task': 'notifications.send_weekly_digest', 'schedule': crontab(day_of_week='monday', hour=8, minute=0), }, } ```

### Database-Defined Schedule (via django-celery-beat)

```python # Manage periodic tasks from Django admin or code from django_celery_beat.models import PeriodicTask, CrontabSchedule import json

schedule, _ = CrontabSchedule.objects.get_or_create( hour='*/6', minute='0', timezone='UTC', )

PeriodicTask.objects.update_or_create( name='Sync inventory every 6 hours', defaults={ 'crontab': schedule, 'task': 'products.sync_inventory', 'args': json.dumps([]), 'enabled': True, } ) ```

## Canvas: Chaining and Grouping Tasks

```python from celery import chain, group, chord

# Chain: run tasks sequentially, passing results pipeline = chain( fetch_data.s(source_id), transform_data.s(), # receives fetch_data result as first arg load_to_warehouse.s(), ) pipeline.delay()

# Group: run tasks in parallel parallel = group( send_welcome_email.s(user_id) for user_id in new_user_ids ) parallel.delay()

# Chord: parallel tasks + callback when all complete result = chord( group(process_chunk.s(chunk) for chunk in data_chunks), aggregate_results.s(), # called with list of chunk results ) result.delay() ```

## Error Handling and Dead Letter Queue

```python # apps/core/tasks.py from celery.signals import task_failure

@task_failure.connect def on_task_failure(sender, task_id, exception, args, kwargs, traceback, einfo, **kw): """Log all task failures to Sentry / alerting.""" import sentry_sdk with sentry_sdk.new_scope() as scope: scope.set_context('celery', { 'task': sender.name, 'task_id': task_id, 'args': args, 'kwargs': kwargs, }) sentry_sdk.capture_exception(exception) ```

```python # Route failed tasks to dead-letter queue after max retries @shared_task( bind=True, max_retries=3, name='payments.charge_card', ) def charge_card(self, order_id: int) -> None: from apps.payments.models import Order, FailedCharge

try: _do_charge(order_id) except Exception as exc: if self.request.retries >= self.max_retries: # Persist to dead-letter table for manual review FailedCharge.objects.create( order_id=order_id, error=str(exc), task_id=self.request.id, ) return # Don't raise — task is permanently failed raise self.retry(exc=exc) ```

## Testing Celery Tasks

### Unit Testing (No Broker)

```python # tests/test_tasks.py import pytest from unittest.mock import patch, MagicMock from apps.notifications.tasks import send_welcome_email

class TestSendWelcomeEmail:

@pytest.mark.django_db def test_sends_email_to_existing_user(self, user): with patch('apps.notifications.services.EmailService') as mock_email: send_welcome_email(user.pk) mock_email.send_welcome.assert_called_once_with(user)

@pytest.mark.django_db def test_skips_missing_user_gracefully(self): """Should not raise when user is deleted between enqueue and execute.""" send_welcome_email(99999) # Non-existent user — must not raise ```

### Integration Testing with CELERY_TASK_ALWAYS_EAGER

```python # config/settings/test.py CELERY_TASK_ALWAYS_EAGER = True # Run tasks synchronously in tests CELERY_TASK_EAGER_PROPAGATES = True # Re-raise exceptions from tasks

# tests/test_integration.py @pytest.mark.django_db def test_registration_triggers_welcome_email(client): with patch('apps.notifications.services.EmailService') as mock_email: response = client.post('/api/users/', { 'email': '[email protected]', 'password': 'strongpass123', })

assert response.status_code == 201 mock_email.send_welcome.assert_called_once() ```

### Testing Retries

```python @pytest.mark.django_db def test_task_retries_on_connection_error(): with patch('apps.crm.services.CRMClient.sync') as mock_sync: mock_sync.side_effect = ConnectionError('timeout')

with pytest.raises(ConnectionError): sync_contact_to_crm.apply(args=[1], throw=True)

assert mock_sync.call_count == 1 # First attempt only when eager ```

## Monitoring

```bash # Inspect active workers and queues celery -A config inspect active celery -A config inspect stats celery -A config inspect reserved

# Check queue lengths (Redis) redis-cli llen celery

# Flower: web-based real-time monitor pip install flower celery -A config flower --port=5555 ```

## Anti-Patterns

```python # BAD: Passing model instances — they may be stale by execution time send_welcome_email.delay(user) # Never pass ORM objects send_welcome_email.delay(user.pk) # Always pass PKs

# BAD: Calling tasks synchronously in production views result = generate_report.apply() # Blocks the request thread

# BAD: Non-idempotent task without guards @shared_task def charge_and_fulfill(order_id): order.charge() # May charge twice if task retries! order.fulfill()

# GOOD: Idempotent with status guard @shared_task def charge_and_fulfill(order_id): order = Order.objects.select_for_update().get(pk=order_id) if order.status != Order.Status.PENDING: return # Already processed order.charge() order.fulfill() ```

## Production Checklist

| Check | Setting | |-------|---------| | Worker restarts on crash | `supervisord` or `systemd` unit | | `CELERY_TASK_ACKS_LATE = True` | Re-queue tasks on worker crash | | `CELERY_WORKER_PREFETCH_MULTIPLIER = 1` | Fair distribution of long tasks | | Separate queues per priority | `-Q default,high_priority,low_priority` | | `CELERY_TASK_SOFT_TIME_LIMIT` set | Graceful timeout before hard kill | | Sentry integration | Capture all `task_failure` signals | | Flower or other monitor | Visibility into queue depths | | Beat runs on single node only | Prevents duplicate scheduled task execution |

## Related Skills

- `django-patterns` — ORM, service layer, and project structure - `django-tdd` — Testing Django models, views, and services - `python-testing` — pytest configuration and fixtures

Intended uses

  • Use Django Celery when this documented workflow matches the task.

Implementation notes

  • Review the source instructions and adapt inputs before running the workflow.

Try it, then tell us how it went

A copied prompt does not tell us whether the task worked. Report your result after checking the output. We collect your tool and outcome, never your files or prompt.