Prompt Engineering for Multi-Step Agents
System prompts, tool schemas, and guardrails that keep agents on-task in production.
System prompt structure
Production agent prompts follow a consistent structure: Role → Constraints → Available tools → Output format → Escalation rules. This is not optional formatting — each section prevents a specific class of failure.
Role defines who the agent is and what it optimizes for ('You are a support agent for Acme SaaS. Your goal is to resolve tickets accurately on the first attempt.'). Constraints set boundaries ('Never share internal pricing. Never delete customer data. Never promise refunds over $100 without escalation.'). Tools tell the agent what it can do with exact names and descriptions. Output format specifies how to structure responses. Escalation rules define when to stop and hand off to a human.
Version-control every prompt change and A/B test against a golden set of 20-50 real tasks. A prompt tweak that improves one workflow often breaks another. Treat prompts like code: review, test, deploy, monitor.
Structured outputs
Force JSON or enum outputs for any step that drives routing, classification, or downstream logic. Free-form text is fine for customer-facing copy, but it is dangerous for decisions like 'should this ticket be escalated?' or 'which department owns this request?'
Use schema validation with automatic retry on parse failure. If the model returns malformed JSON, retry with the error message appended ('Your previous response was invalid JSON: ...'). After two failures, escalate to a human or a safe default.
Define enums explicitly. 'Respond with exactly one of: RESOLVE, ESCALATE, NEEDS_INFO.' Open-ended classification ('determine the appropriate action') leads to inconsistent routing that erodes customer trust. Specificity in prompts directly translates to reliability in production.
Failure modes
Three failure modes kill agent products in production: hallucinated actions, infinite loops, and runaway tool costs.
Hallucinated actions happen when the agent claims it sent an email or updated a record but never called the tool. Fix: require tool call evidence before reporting completion ('You must call send_email and receive a success response before telling the user the email was sent.'). Infinite loops happen when the agent retries the same failing step. Fix: max iteration count (10-15) with forced escalation. Runaway tool costs happen when the agent calls expensive tools repeatedly. Fix: per-task tool call budgets and cost tracking.
Log every prompt, tool call, and response with a task ID. When a customer reports a failure, you need to replay the exact run in under 5 minutes. Debugging agents without traces is like debugging code without a stack trace — technically possible but painfully slow.