Your API wrappers, data transformers, and tool implementations should have deterministic unit tests — same input always produces same output. Run these in CI on every commit and catch integration bugs before they reach production.

Mock LLM responses in CI tests. Create fixtures for common agent scenarios: 'given this ticket input, the agent should call get_customer, then search_kb, then draft_response.' Test that the right tools are called in the right order with the right parameters. The model's creative output is irrelevant — the orchestration logic is what you test.

Run live LLM evals nightly or on-deploy, not in CI. Live evals cost money and have variable results, but they catch prompt regressions that mocked tests miss. Separate fast deterministic tests (CI, every commit) from slow probabilistic evals (nightly, pre-deploy).

Using an LLM to score agent outputs is useful but dangerous if it is your only quality gate. LLM judges inherit the biases of the judge model and can be gamed by agents that produce verbose, confident-sounding wrong answers.

Build rubric-based scoring with human-labeled calibration examples. Define explicit criteria: 'Accuracy: did the agent use correct information? (1-5). Completeness: did it address all parts of the request? (1-5). Tone: is it appropriate for the context? (1-5).' Calibrate the LLM judge against 50+ human-labeled examples until its scores correlate with human scores above 0.85.

Never rely solely on LLM-judge pass rates. Always include human review for a sample of production outputs (5-10% initially, decreasing as confidence grows). LLM-as-a-judge is a screening tool, not a replacement for human judgment.

Before promoting a new agent version to production, run it in shadow mode: process the same inputs as production, compare outputs, but do not show shadow results to users. Promote the new version when shadow metrics beat production baseline for at least a week.

Track shadow vs. production on: completion rate, escalation rate, average latency, cost per task, and human approval rate (for agents with human checkpoints). A new version that completes 5% more tasks but costs 3x per task is not an improvement.

Shadow mode is especially critical for prompt changes. A prompt tweak that improves response quality on 90% of tasks might break the 10% of edge cases that matter most to your best customers. Shadow mode catches these regressions before they affect real users.