Metrics Dashboard: What to Track from Day One
Task success rate, cost per task, time saved, and the KPIs that prove ROI to customers.
Product metrics
Track five product metrics from your first deployment: task completion rate (% of started tasks that finish successfully), escalation rate (% handed to humans), average steps per task (efficiency indicator), latency p95 (user experience), and cost per completed task (unit economics).
Segment every metric by customer, workflow, and time period. Aggregate numbers hide problems — a 90% completion rate might mask one customer at 50% and another at 99%. Per-customer segmentation enables proactive support and identifies your best case studies.
Set alert thresholds: completion rate drops below 80%, escalation rate exceeds 25%, p95 latency exceeds 30 seconds, or cost per task exceeds your pricing floor. These alerts catch problems before customers notice and complain.
Business metrics
Standard SaaS metrics apply: MRR, churn rate, net revenue retention, customer acquisition cost (CAC), and lifetime value (LTV). For agent products, add one critical metric: inference cost as a percentage of revenue.
Target inference costs under 20% of revenue at scale. During early pilots, 30-40% is acceptable while you optimize. Above 50% means you are subsidizing customer usage and need to fix pricing or architecture before scaling sales.
Calculate LTV with inference costs included, not just hosting and support. An agent customer at $500/month with $150/month inference cost has a very different LTV than one with $50/month inference cost. Gross margin after inference is the metric that determines whether your business model works.
Customer-facing ROI
Build a dashboard your customers can show their boss — this is your most powerful retention and expansion tool. Display: tasks completed this month, estimated hours saved (tasks × average manual time), success rate trend, and cost comparison vs. their previous manual process.
Make the ROI math transparent and conservative. 'Estimated 120 hours saved this month (based on 400 tasks × 18 min average manual time)' is believable. 'Saved $50,000 this month!' is not.
Customers who can articulate ROI to their leadership renew and expand. Customers who cannot explain why they pay you churn at the first budget review. Your ROI dashboard is not a nice-to-have — it is your retention strategy. Update it weekly and include it in every QBR.