TutorialsPromptFoo Editorial9 minUpdated May 25, 2025

How to Write an AI Workflow That Doesn't Break in Production

Six lessons from workflows that survived contact with real traffic — retries, schema validation, cost caps, and human gates.

Idempotency first

A workflow that can run twice on the same input without side-effects is a workflow you can debug. Design keys before you design steps.

Validate every output

Every LLM output is untyped text until you validate it. Enforce a JSON schema and route validation failures to a fix-and-retry path.

Retries with backoff

External APIs fail. Retry with exponential backoff on 5xx and rate limits, but never on 4xx caused by your input.

Cost caps

Add a per-run and per-day spend cap. A prompt that loops on itself can burn a month of budget in an hour.

Human gates for high-stakes actions

Sending email, moving money, or writing to production databases should route through an approval step for anything above a threshold.

Observability

Log inputs, prompts, model, tokens, latency, and outputs to something you can query. You cannot fix what you cannot see.

FAQs

Related resources