How to Evaluate Agent Ideas Before You Build
A scoring framework for market size, build difficulty, defensibility, and willingness to pay.
The four filters
Run every idea through four filters before writing code. Missing even one makes go-to-market 10x harder.
(1) Painful, frequent workflow - does this task happen daily or weekly, and do people complain about it? 'Annual compliance report' fails the frequency test. 'Daily ticket triage' passes. (2) Measurable ROI - can you quantify hours saved, revenue gained, or errors prevented? Vague value props ('better insights') do not close deals. (3) API-accessible data - can you read and write the systems involved programmatically? PDF-only workflows with no APIs require OCR hacks that break constantly. (4) Buyer with budget - is there a person whose job performance is measured by this workflow's outcome?
Score each filter 1-5. Ideas scoring below 14/20 need a narrower wedge or a different target customer. Ideas scoring 18+ are strong candidates for a first product.
Market score explained
Our database scores ideas on four dimensions: demand signals (search volume, community discussions, existing manual workarounds), competition (how many funded startups and incumbents are in the space), integration moats (how long deep integrations take to build), and pricing power (what buyers currently pay for alternatives).
Scores above 8.0 indicate strong market pull with reachable buyers - these are good first products. Scores of 6-8 benefit from a niche wedge: pick one sub-industry, one company size, or one integration to start narrow and expand later. Scores below 6 often mean the market is too early, too crowded, or too hard to reach profitably.
Use scores as a starting filter, not a final decision. A score of 7.5 in a vertical where you have domain expertise and warm leads beats a 9.0 in a space where you know nobody. Your unfair advantages matter as much as market size.
4 more sections in this article
- Validate in 48 hours
- Score honestly, or do not score
- Signals that look like demand but are not
- When to kill an idea
This is Pro content
Get Agent Factory Pro - a one-time payment for lifetime access to full articles, complete build prompts, and everything new.
Use it
The parts of the library that put this article to work.
Published 27 July 2026. Last reviewed 17 August 2026. We re-read this library on a schedule and date every article, so you can see for yourself how current it is.