How to Build Idempotent Cloud Tasks Handlers in Python

The Problem

Google Cloud Tasks guarantees at-least-once delivery. That means your handler WILL be called multiple times for the same task. If you're not handling this, you have bugs.

# BAD: This creates duplicates on retry
@app.post("/tasks/score")
async def handle_score_task(request: ScoreRequest):
    result = ScoringResult(
        id=uuid4(),  # New ID every time!
        evaluation_id=request.evaluation_id,
        score=compute_score(request),
    )
    db.add(result)
    db.commit()

Cloud Tasks retries if your handler returns 5xx, takes longer than the timeout, or the network hiccups. Every retry creates a new row with a new UUID.

The Fix: 3-Step Idempotency Pattern

Step 1: Deterministic IDs

import hashlib

def compute_task_id(evaluation_id: str, model: str, turn: int) -> str:
    payload = f"{evaluation_id}:{model}:{turn}"
    return hashlib.sha256(payload.encode()).hexdigest()[:32]

Step 2: Guarded Upserts

INSERT INTO scoring_results (id, evaluation_id, score, model, turn)
VALUES ($1, $2, $3, $4, $5)
ON CONFLICT (id) DO NOTHING
RETURNING id;

Step 3: Always Return 200

If your handler detects a duplicate and returns 409 (Conflict), Cloud Tasks will retry it. Forever. Always return 200 for idempotent operations.

Handling Multi-Step Tasks

Each step gets its own deterministic ID derived from the parent task ID. If the handler crashes after step 2, the retry skips steps 1 and 2 (already exist) and only executes step 3.

Cloud Run Configuration

# Don't overwhelm the DB
containerConcurrency: 10  # Not 80
timeoutSeconds: 3600      # Match your longest task

We reduced concurrency from 80 to 10 per instance. Each handler holds a database connection. 80 concurrent handlers × 10 instances = 800 connections. Our Cloud SQL instance can't handle that.

Key Takeaways

Problem	Solution
Random UUIDs	Deterministic IDs from payload
INSERT	INSERT ... ON CONFLICT DO NOTHING
Return 409 on duplicate	Return 200 always
No logging	Correlation IDs from Cloud Tasks headers
High concurrency	Cap containerConcurrency (10, not 80)

This pattern handles our entire evaluation pipeline — thousands of tasks per day, zero duplicates, zero data corruption.