Prompt Engineering for Multi-Step Agents
System prompts, tool schemas, and guardrails that keep agents on-task in production.
System prompt structure
Production agent prompts follow a consistent structure: Role → Constraints → Available tools → Output format → Escalation rules. This is not optional formatting - each section prevents a specific class of failure.
Role defines who the agent is and what it optimizes for ('You are a support agent for Acme SaaS. Your goal is to resolve tickets accurately on the first attempt.'). Constraints set boundaries ('Never share internal pricing. Never delete customer data. Never promise refunds over $100 without escalation.'). Tools tell the agent what it can do with exact names and descriptions. Output format specifies how to structure responses. Escalation rules define when to stop and hand off to a human.
Version-control every prompt change and A/B test against a golden set of 20-50 real tasks. A prompt tweak that improves one workflow often breaks another. Treat prompts like code: review, test, deploy, monitor.
Structured outputs
Force JSON or enum outputs for any step that drives routing, classification, or downstream logic. Free-form text is fine for customer-facing copy, but it is dangerous for decisions like 'should this ticket be escalated?' or 'which department owns this request?'
Use schema validation with automatic retry on parse failure. If the model returns malformed JSON, retry with the error message appended ('Your previous response was invalid JSON: ...'). After two failures, escalate to a human or a safe default.
Define enums explicitly. 'Respond with exactly one of: RESOLVE, ESCALATE, NEEDS_INFO.' Open-ended classification ('determine the appropriate action') leads to inconsistent routing that erodes customer trust. Specificity in prompts directly translates to reliability in production.
5 more sections in this article
- Failure modes
- Writing the role and the constraints
- Tool descriptions are part of the prompt
- Guardrails that actually hold
- Treat prompts like code
This is Pro content
Get Agent Factory Pro - a one-time payment for lifetime access to full articles, complete build prompts, and everything new.
Use it
The parts of the library that put this article to work.
2 more build prompts reference this article.
Published 27 July 2026. Last reviewed 17 August 2026. We re-read this library on a schedule and date every article, so you can see for yourself how current it is.