What I'd do with it
- Open every account with the effort-level test.Customers pick an effort level on day one, usually by guessing. Too low and answers go quietly wrong; too high and the bill is five to seven times what the job needs. A sample of their own questions settles it, and the number to bring back is cost per right answer.
- Call before the invoice does.Section 06 lists five things a new developer runs into alone, including two lines on the bill that are easy to miss. Each shows up in usage before it shows up as a complaint, which makes it a reason to pick up the phone.
- Sell the layers as one account.The agent is the easy way in. The search and the models underneath are on the same key, so a customer who wants more speed or control doesn't need a second vendor.
- Right-size the level, then grow the volume.Telling a customer the cheap level is enough earns the trust. The contract comes from how much they run and from the research jobs that need the higher levels. The table below shows what that takes.
What $100,000 a year looks like in usage
At the prices I measured, before any volume discount| Job | Measured cost | Per year | Per day |
|---|---|---|---|
| Search requests | $5.00 per 1,000 | 20 million | 55,000 |
| Company records, fast | $4.10 per 1,000 | 25 million | 68,000 |
| Multi-step answers, low | $5.20 per 1,000 | 19 million | 53,000 |
| Company records, medium | $12.50 per 1,000 | 8 million | 22,000 |
| Research briefs, high | 12 cents each | 860,000 | 2,400 |
Why I ran it
Nobody demos an API. A developer signs up with a credit card and tries it the same day. The contract comes later, when that developer's team runs a sample of its own work through the product and comes back with three numbers: how often it was right, what it cost, and whether it broke. Those numbers decide the deal, usually without a seller in the room.
I wanted those numbers first, so I became a customer: a new account, a credit card and $3.60 of usage. The product is the Agent API. You send a question; it searches the web, reads what it finds and writes an answer with sources. A dial sets how hard it works, and what you pay, from fast to xhigh.
I tested everyday jobs, not hard research: looking up a fact, combining several facts, filling in company records for a CRM, and handling questions about events that never happened. Every answer was graded against a key pulled from the original source.
Match the effort to the job
The first set was 15 lookups with one right answer: the latest release of nine widely used software projects, and the closing price of six stocks on November 19. Every effort level got all 15 right. The cheapest did it in 1.7 seconds.
The second set was 10 questions that need several facts combined. Which of six stocks moved most on a given day? How many releases did a project ship in October? Here the cheapest level got 6 right, and the next level up got all 10.
Accuracy and cost by effort level
Agent API, answers graded against a key pulled from the original sourcesSimple lookupscorrect, of 15
Multi-step questionscorrect, of 10
Cost per correct multi-step answerUS cents, lower is better
The fast level's mistakes were confident and plausible. Asked which of six stocks had the largest move on November 19, it answered CrowdStrike, up 2.71%. That figure is real: CrowdStrike was the biggest gainer. The right answer was NVIDIA, which fell 3.30%. Fast answered the easier question it found. Low ran more searches and caught it, at about half a cent per right answer. High was right every time too, at five times the price.
Nobody would notice the CrowdStrike answer was wrong without checking it. That is the risk of choosing a level by price, and the reason to choose it from a sample of the customer's own questions.
Filling in records, and refusing to make things up
A common paid use is enrichment: hand the agent a company name and get back a filled-in record for a CRM. I asked for seven fields on 20 public companies (ticker, headquarters, CEO, headcount, fiscal year, revenue and head of security), returned in a fixed format.
Company records, fast vs medium effort
20 public companies, 7 fields each| Measure | Fast | Medium |
|---|---|---|
| Complete records in the requested format | 20 of 20 | 20 of 20 |
| Revenue and CEO | Same answer at both levels | |
| Head of security named | 12 of 20 | 16 of 20 |
| Typical time per record | 3.3 s | 12.0 s |
| Cost per 1,000 records | $4.10 | $12.50 |
The public facts were settled at the cheap level. Medium cost three times as much, and its extra time went into the one field that is hard to find, the head of security, where it named four more people. That field still needs a human check before anyone emails it.
The last set tested trust. I asked 12 questions built on events that never happened, such as how much Apple paid to acquire Netflix, or who won the 2024 Nobel Prize in Mathematics. At three effort levels, all 36 answers rejected the premise.
"The Philadelphia Eagles won Super Bowl LIX, defeating the Kansas City Chiefs 40-22. It was played in New Orleans, not San Diego."Fast effort, asked "Which team won the 2025 Super Bowl played in San Diego?"
For enrichment, the cheap level is the product and the higher level is an upsell for the hard fields. "Does it make things up?" is the first question a risk team asks. Here the answer was no at every price, on famous cases; a customer should test its own.
What depth costs on a real research job
The jobs above were small, and even the high level stayed near two cents. A research brief is different. I asked for a one-page brief with sources on three public companies. At fast it cost half a cent, took 9 seconds and leaned on the company's homepage. At high it cost 10 to 13 cents, took over a minute, and found the annual report, job postings and trade press. Low was enough for everyday questions; research is where the higher levels earn their price.
Every response states its own cost in dollars, step by step, so a developer can forecast spend without talking to anyone. Two lines on that bill are easy to miss when you budget from the headline prices. Reading a web page is listed at a twentieth of a cent; on my bill each read cost between half a cent and 1.6 cents, because the model's work on the page is charged with it. The pricing page says so, in one sentence. And at high effort about 60% of each bill was the charge for storing work so later steps of the same request can reuse it.
Underneath the agent: search and models
The same key also sells the two parts the agent is built from. I tested each, then both together.
Search. To get readable text from the top five web pages for a question, my own setup at home makes six requests, and sites refused a quarter of them. The Search API does it in one request, in a fifth of the time, with usable text from almost every source.
Getting usable web text: my setup vs the Search API
32 questions, top 5 sources eachTypical time to usable textseconds, lower is better
Sources with usable textpercent, higher is better
Questions left shortunder 3 usable sources, of 32
Models. Orchestrator, launched November 17, puts models from several labs, plus open models Perplexity hosts itself, behind one account. I gave seven of them the same job: label 30 real news headlines positive, negative or neutral. At default settings all seven gave the same label on 27 of the 30. Yet the cheapest cost a cent and a half per thousand headlines and the most expensive $3.25, over 200 times more, by my calculation from token counts and list prices. Part of that gap is a default: many models write hidden working notes before answering, and those notes are billed. The most expensive model added about 1,200 tokens of its own to every request.
Both together. I then built my own answer service from the parts: the Search API feeding a hosted model, on the same key. Across 12 questions, a full answer with sources came back in 1.6 seconds, against 3.5 for the agent at its fastest level. It cited four of the five sources it was given; the agent's answers linked two or three. It cost about twice as much, and most of the bill was the search text going into the model.
The agent is the easy way in. The parts underneath are where a growing account goes for more speed or control over cost, on the same key and the same bill. A vendor that sells only search, or only models, can't offer that.
What a new developer runs into
None of these is serious alone. Together they are what a self-serve developer meets before anyone at the company knows they exist. I'd take them to the product team, and I'd use them as reasons to call.
| What they hit | What it costs them |
|---|---|
| No guide to which effort level fits an everyday job | They guess. Too low loses accuracy; too high costs 5-7x. |
| Two lines on the bill are easy to miss | A page read costs 10 to 30 times its headline price once the model's work is added, and at high effort a storage charge is the largest line. Both are documented; neither is in the headline price. |
| The docs don't say how sources come back | The guide's examples hint at numbered markers; live answers used plain links. My first parser found none. |
| New accounts are limited to one request per second | The first real test hits the limit, and some models stay locked until $50 is spent. |
| Models write billed working notes by default | A simple job moved to a new model can cost 8x what it needs to, and the off switch doesn't work on every model. |
Method and limits
All tests ran November 19-20, 2026, from one server in San Diego on a new account's starting tier.
- Small samples, as stated in each section, each run once. The test sets are mine, not a standard benchmark.
- Costs for the Agent API come from each response's own cost field. Search and model costs are my calculation from request and token counts at list price. Not yet matched to an invoice.
- Not graded for accuracy: head-of-security names, research briefs, and the search-plus-model answers. The model comparison is scored against the group's majority, not a human key.
- Not tested: the xhigh level, OpenAI and Anthropic models (locked on a new account), Orchestrator's automatic failover between providers, other vendors' products, and load above one request per second.
- This is a new developer's first two days. It says nothing about what an enterprise signs on: volume limits, uptime commitments, data handling or contract pricing.