The AI Queue & Background Job Migration Plan

1. The Synchronous Timeout Trap

When an endpoint performs heavy computation or calls external APIs directly during an HTTP request, it relies on a simple assumption: every hop between the browser and your code stays connected until the server finishes. In production, something upstream gives up first. Nginx closes an idle upstream read after its default proxy_read_timeout of 60 seconds. Cloudflare returns a 524 when the origin sends no response within its default proxy read timeout of 125 seconds, adjustable only on Enterprise plans. Managed platform routers and load balancers add their own ceilings, often tighter than both.

Do not tune your architecture to a specific number. The number is whichever limit in the chain is smallest, it is set by infrastructure you may not control, and it changes when someone puts a new proxy in front of the app. Work that has no bounded upper duration does not belong in a request-response cycle at any timeout value.

When you prompt an AI coding assistant to fix a request timeout, its default impulse is often to increase server timeout limits or wrap the execution in a quick setTimeout or fire-and-forget background promise. Fire-and-forget logic inside a web process is dangerous: if the server restarts or deploys a new release, in-flight jobs vanish without a trace.

Moving to a dedicated message queue (such as Redis/BullMQ, SQS, or Celery) decouples the web tier from heavy execution. However, extracting synchronous code into an asynchronous worker alters how errors, retries, and return values behave.

2. Phase 1: Payload Isolation & Job Dispatch

The first migration phase requires refactoring the API endpoint so it only validates inputs, creates a tracking record in the database, enqueues a job payload, and immediately returns a 202 Accepted response to the client.

A common mistake when instructing AI to write the dispatch code is passing entire database objects or large file blobs directly in the queue message. Queue payloads should remain lightweight references.

// Good queue payload
{
  "jobId": "job_984f1a20",
  "documentId": "doc_102",
  "requestedBy": "usr_402",
  "attempt": 1
}

If you pass full records in the queue payload, the worker may process stale data if the record was updated between enqueue time and execution time. Pass entity IDs so the background worker fetches the latest state directly from the database upon execution.

3. Phase 2: Mandatory Idempotency & State Checks

Distributed queues guarantee at-least-once delivery. Network blips or worker restarts mean the same job payload will eventually run more than once. If worker execution is not idempotent, users get double-billed, duplicate emails are sent, or corrupted records are written to the database.

Before letting an AI assistant write worker handler functions, require it to include an idempotency check and an explicit state machine transition:

• Pending: Job is enqueued, database record created.
• Processing: Worker acquires lock / updates status to processing.
• Completed: Output saved, status updated to completed.
• Failed: Error logged, status marked as failed or queued for retry.

If a worker receives a job whose status is already Completed or currently Processing under an active lock, it should log a duplicate warning and return cleanly without re-executing the heavy side effects.

4. Phase 3: Retry Policies & Dead-Letter Queues (DLQ)

AI-generated workers frequently lack granular error categorization. If an external API returns a 401 Unauthorized error due to invalid credentials, retrying the job 10 times with exponential backoff will only flood logs and burn rate limits.

Distinguish between transient errors (network timeout, rate limit 429, temporary DB connection loss) and fatal errors (invalid schema, malformed payload, 401/403 auth failure). Transient errors should trigger retries with jittered exponential backoff; fatal errors must fail fast and move directly to a Dead-Letter Queue (DLQ).

try {
  await processDocument(job.data.documentId);
} catch (error) {
  if (isTransientError(error)) {
    // Retry with backoff
    throw error;
  } else {
    // Non-retryable error: record failure and move to DLQ
    await markJobFatal(job.data.jobId, error.message);
    return;
  }
}

5. Phase 4: Frontend State Visibility & Polling

When an API endpoint changes from returning final data to returning 202 Accepted with a jobId, the frontend UI must adapt. Asking AI to rewrite the frontend without guidelines often leads to aggressive 500ms polling loops that overwhelm web servers.

Provide explicit rules for the frontend client implementation:

• Initial Response: Show immediate optimistic pending state in the UI.
• Polling Schedule: Use adaptive polling (e.g., poll every 2s for 10s, then back off to 5s, timing out after 2 minutes).
• WebSockets / SSE: For real-time applications, trigger server-sent events or WebSocket broadcasts upon worker state changes instead of polling.
• Terminal States: Ensure both successful output rendering and actionable error banners are handled gracefully.

6. Phase 5: Verification & Cutover Runbook

Before deploying your queue migration to production, test the async architecture under real stress:

1. The Kill Test: Force-kill a worker process halfway through execution. Verify that another worker picks up the job and that idempotency guards prevent duplicate side effects.
2. The Backpressure Test: Enqueue 1,000 jobs simultaneously. Verify that worker concurrency limits protect downstream databases from connection pool exhaustion.
3. The Poison Pill Test: Submit an intentionally malformed payload. Verify that it routes to the DLQ after configured retries without halting the queue processor.

With idempotency keys in place, transient retry boundaries defined, and user feedback mechanisms active, migrating synchronous endpoints to background queues becomes a controlled, repeatable architectural upgrade.

Copy-Paste Prompt: Queue Extraction Analysis

Resolve the assumptions before asking for an implementation. Then migrate one endpoint at a time and keep the synchronous path available behind a flag until the queued path has survived the three tests above under real load.