Agent tasks often take 5-30 seconds — an eternity in UI time. Streaming transforms this from 'is it broken?' to 'I can see it working.' Stream tokens as they generate, and stream tool status updates ('Searching knowledge base...', 'Found 3 results...', 'Drafting response...').

Perceived speed matters more than actual speed. An agent that streams progress updates at 15 seconds feels faster than one that returns a complete response at 10 seconds with no feedback. Users tolerate latency when they understand what is happening.

Implement streaming at two levels: token streaming for text generation (standard SSE/WebSocket) and event streaming for tool calls and status changes. The Vercel AI SDK, LangChain, and most agent frameworks support this. Make streaming the default UX, not an add-on feature.

Tasks that take more than 30 seconds should not block the UI. Move them to a background queue with email or webhook notification on completion. Report generation, batch processing, multi-document analysis, and complex research tasks all belong in async queues.

Common choices: Inngest (serverless-friendly, great DX), Trigger.dev (similar, strong TypeScript support), BullMQ (self-hosted Redis queue, more control). The pattern is the same: enqueue job → return job ID immediately → process in background → notify on completion.

Show job status in the UI: queued, running (with step progress), completed (with results), failed (with error and retry option). Users who submit a 5-minute analysis job and get no feedback for 5 minutes assume it failed. Progress updates retain users through long-running tasks.

Per-tenant concurrency caps protect your API budget and prevent one customer from starving others. Without limits, a single customer running batch jobs can consume your entire OpenAI quota and degrade service for everyone else.

Implement limits at three levels: requests per minute (prevent burst abuse), concurrent agent runs per tenant (prevent resource hogging), and daily/monthly task quotas (align with pricing tiers). Start generous and tighten based on usage data.

When a customer hits a limit, show a clear message with upgrade path: 'You have reached your plan limit of 500 tasks/month. Upgrade to Pro for 2,000 tasks/month.' Rate limits are a pricing feature disguised as infrastructure — they naturally drive upgrades from power users.