Webhooks time out. LLM workflows don't care.

On the AI interview platform I work on, an interview ends and an LLM evaluates it. That evaluation takes a while: transcripts are long and prompts are chained. The trigger was a webhook. The webhook had a timeout.

You can guess the rest.

The failure mode

A synchronous webhook says: call me, wait, and I’ll answer with the result. That’s fine when the work takes 200 milliseconds. It falls apart when the work is an LLM pipeline whose duration depends on transcript length, provider latency and whether a retry fires.

What we saw:

• Requests that sometimes finished and sometimes hit the timeout — with no change in input quality.
• The caller giving up while the workflow kept running, so results were produced but never delivered.
• Retries from the caller starting duplicate evaluations of the same interview.

None of these are model problems. They are plumbing problems.

The fix: acknowledge first, work later

I redesigned the workflow in n8n around one rule: the webhook never waits for the LLM.

1. Accept. The webhook validates the request, records a job, and responds immediately with 202 Accepted and a job id.
2. Process. A separate execution picks up the job and runs the long evaluation chain at its own pace.
3. Deliver. When the evaluation is done, the result is written back and the report becomes available — the caller never had to hold a connection open.

caller ──POST──▶ webhook ──▶ save job (pending) ──▶ 202 { jobId }
                                  │
                                  ▼
                        async execution: evaluate
                                  │
                                  ▼
                     save result (done) ──▶ report ready

Things worth getting right

• Idempotency. Key jobs on the interview id so a retried request finds the existing job instead of starting a new one.
• Explicit status. pending → running → done | failed makes the system debuggable and the UI honest.
• Failures are data. A failed evaluation should be a row you can inspect and re-run, not a log line you hope someone reads.

If a step’s duration depends on a model, it doesn’t belong inside a request/response cycle.

The result was boring in the best way: timeouts stopped being a failure mode, and the workflow became reliable. Most “AI reliability” work looks like this — not smarter prompts, just better plumbing.