20 Oct 2026, 1:45 to 2:01 PM PT Netflix posted Q3 results at about 1:01 PM. Then we asked nine sources.
The answer, from the SEC filing: revenue grew 12% to $12.9B .
ExaAI native search "Netflix reports results for third quarter of 2026" , then the SEC filing, then CNBC from today
the number
LinkupAI native search "Netflix Q3 Revenue Rises 12% to $12.87 Billion"
the number
ParallelAI native search CNBC from today, then "Netflix Q3 2026 Earnings: Revenue of $12.87 Billion"
the number
TavilyAI native search "Netflix Q3 2026 Earnings Preview" : analysts expected $12.9B
the forecast
Claude WebSearchmodel + search tool The SEC filing itself. Its summary: "Revenue grew 12%" , no dollar figure
the filing, no number
SerperGoogle results via API Netflix's investor page, then "Analysts expect ... revenue of $12.88 billion"
a "results coming" page
Firecrawl searchsearch + scrape The same pages as Serper, in the same order
a "results coming" page
Googlescraped via Apify Wikipedia "2026", the October Visa Bulletin, a Nobel press release
nothing on Netflix
Wikipediasite search The Trade Desk, Big Tech, Roku
nothing yet
3 had the number 1 the forecast 1 the filing 2 a "results coming" page 2 nothing
research/battle/results_fresh/<source>.jsonl id fr05 · gold: sec.gov ex991_q326.htm (8-K exhibit 99.1)
Netflix posted its third quarter results today (20 Oct) at about one o'clock. About fifty minutes later we typed this into nine search sources.
Press by press: three had the actual number. Tavily had the analyst forecast, right number, wrong reason. Claude found the SEC filing but its summary dropped the dollar figure. Two had pages announcing the announcement. Two had nothing.
Same question. Nine different answers. Evidence: results_fresh, id fr05.
One of four talks
Four talks, one question: how should agents search?
A Search API Battle Royale Same questions, different sources, measured 20 Oct 2026.
B System 1 vs System 2 search Quick search, deep research, and when to route between them.
C Self improving RAG A research loop that learns from its own mistakes.
D Agent native search platforms What search looks like when the user is an agent.
Four talks share one question; this one is lit. Each stands alone. Together they cover where the facts come from, how hard to search, how a research pipeline learns, and what search built for agents looks like.
Claude Code Build Night Bright Data SF · 20 Oct 2026 · Ray Zahid, Immersive Commons
Search API Battle Royale
We tested nine search sources on 75 questions with known answers. All on 20 Oct 2026.
research/battle/summary_all60.json (60 trivia) · summary_fresh.json (15 news) · picked_contested.json (10, no answer key)
I am Ray. I run Immersive Commons, an AI builder floor in Frontier Tower.
This afternoon (20 Oct) we ran the same questions through every search source we could sign up for. Sixty trivia questions, fifteen news questions from last week, and ten questions where sources might slant.
Not a league table. A snapshot, and what it tells you about building an agent.
The question of this talk
What you know depends on what knowledge your agents can fetch .
How can you build a strategy for search?
One line for the whole talk. Your agent does not know things. It reads what a search source hands it.
Press. So the question is: how do you build a strategy for search? Which source, when it matters, and how your agent can tell it got bad results. We will answer all three with numbers from today (20 Oct).
1
Part 1 · Foundations
How a search engine works
Before any results, the parts. Every search engine does four things: it crawls, it indexes, it ranks, and it writes a snippet. Each step quietly decides what your agent can know.
1.1 Crawl, index, rank, snippet
Four steps between the web and your agent
The web pages change all day Crawl a robot reads pages decides how new
Netflix went live at 1:01 PM.
No new visit yet: not found.
Index the catalogue decides what can be found
Semantic Scholar: papers only.
0 of 30 answers found.
Rank who goes first what is read first
First place, of 60 trivia:
Exa, Tavily 32. Google 22.
Snippet the lines under a link what can be quoted
133 to 1,534 characters
per result, by source.
Your agent often reads only this the top 5 results,
a wait and a bill
results_fresh/*.jsonl fr05 · summary.json s2 (results for 4 of 30, 0 answers) · summary_all60.json answer_in_top1, mean_snippet_chars
Press one. Crawl: a robot visits pages and follows links. How often it comes back decides freshness. Netflix posted at one o'clock; an engine that had not come back could not know.
Press two. Index: the catalogue. The engine only finds what it already wrote down. Semantic Scholar holds papers, so trivia gets nothing.
Press three. Rank: who goes first. Press four. Snippet: the few lines under each link. Often the only part your agent reads.
1.2 Scrape it, rent a scraper, or use a new index
Three ways to get search results
1 · Scrape a results page
Load the page
how load the page and copy out the links we used Google, through an Apify scraper catch slow (5.62 s median), breaks when the page changes 2 · Search results API
Rent a scraper
how a company scrapes the page and sends you the results as data we used Serper. Also SerpAPI and Bright Data catch short text: 144 characters per result on average 3 · AI native search API
Built for AI
how its own copy of the web, long text, full pages on request we used Exa, Tavily, Linkup, Firecrawl search catch each one sees a different part of the web Claude's WebSearch goes one step further: links plus a summary a model already wrote.
summary_all60.json median_latency_s (google, amortised over a batch), mean_snippet_chars (serper) · content/battle.md C5
Three ways to get results. One: scrape the search page yourself. Slow and fragile. Two: a search results API, a company that scrapes for you and hands back the results as data. Tonight's host sells one. Three: AI native search, with its own index, built to feed a model.
Press. Claude's own WebSearch tool goes one step further: it hands back links and a summary a model already wrote.
1.3 Words we use tonight
Six terms we use tonight
SERP
The results page.
A ranked list for one search: a title, a link and a snippet for each.
Snippet
The lines under the link.
Text picked to match your search. A results page gives about 150 characters.
Coverage
Does the source have anything on this?
How many questions get results, and how many get the answer.
Freshness
How up to date it is.
The lag between the world changing and the index changing.
Ranking
Who goes first.
A score from matching words, links, newness and trained models.
Source bias
Which kinds of sites it favours.
Which websites, views and languages show up most.
Six words, one per press. SERP is the results page. A snippet is the lines under each link. Coverage: does the source have anything at all. Freshness: how far behind the world it is. Ranking: who goes first. Source bias: which kinds of sites it puts in front of you.
Every result in this talk is one of these six.
1.4 Three meanings of fair
"Fair" means three different things
A fair test We kept the questions, grader, model and prompt the same for every test.Fair access Who can use the source at all, and on what terms.free sources with no sign-up broke first Fair representation Whose pages the engine puts in front of your agent: papers, forums, review sites.10 questions people argue about
We keep the three apart. A source can win one and lose the other two .
When people say a comparison is fair, they mean one of three things. A fair test: we kept the questions, the grader, the model and the prompt the same. One thing we did not make equal is how much text each source sends. That comes back later.
Fair access: who can use it at all. Fair representation: whose pages your agent sees. We measured all three today (20 Oct), roughly.
2
Part 2 · The test
The test
The sources, the questions and the scoring.
Now the test. Who was in it, what we asked, and the one grader every source faced.
2.1 The contestants · method
Nine sources answered every question
Tested on all 85 questions
Googlescraped via Apify
SerperGoogle results via API
Firecrawl searchsearch + scrape
TavilyAI native search
ExaAI native search
LinkupAI native search
ParallelAI native search
Claude WebSearchClaude Code's search tool
WikipediaWikipedia's own search
Stopped after 30: blocked or no results
DuckDuckGo Lite, Semantic Scholar, OpenAlex, our own research library Ran, results held back
Bright Dataits MCP search tool, Google engine · our code mixed up its replies Described, not tested
Jinasign-up did not complete
Google Custom Searchunavailable to new customers
These numbers hold for our questions, on 20 Oct 2026 only.
research/battle/ACCOUNTS.md (free tiers, no card) · TOS_PRICING.md (terms read 20 Oct 2026) · summary.json (first 30)
Nine sources ran every question. New free accounts, no payment details. Google came in two ways: scraped through Apify, and as an API through Serper.
Press. Four sources only ran the first thirty and broke or came back empty. Tonight's host, Bright Data, ran on all 85 questions, but our own client paired some replies with the wrong questions, so we hold its numbers back until that is fixed. Jina's sign-up did not complete, so we describe it and do not score it.
Press. Read every number as: our questions, one afternoon.
2.2 Three question sets, one grader
Question set and scoring rubric
60 Trivia 30 real Google searches and 30 harder questions that need two facts. Each has a known answer.
15 Last week's news From 13 to 20 Oct 2026. Each has a dated page that quotes the answer.
10 Contested Questions people argue about: best VPN, rent control, coffee and the heart. No right answer, so we look at which sites come back.
One search the question as typed
Top 5 titles and snippets only
Found score is the known answer in those five results?
Answer score Claude Sonnet answers using only those five. Same check.
picked.json + picked_rest.json (eval/set.jsonl, seed 20261020) · picked_fresh.json · picked_contested.json · eval/grade.py contains_gold, correct
Three sets. Sixty trivia questions with known answers. Fifteen questions about last week's news, written today (20 Oct) with a dated source for each. Ten contested questions with no answer key.
Press. One query per source, as typed. Top five only. Two scores: is the answer in the five, and does a model answer correctly from them.
3
Part 3 · Results
Results Did it find the answer, how fast, how different, and did it help the model answer?
Results. First: is the answer even in the top five. Then speed, then how different these sources really are, then news, then whether any of it changes the model's answer.
3.1 Round 1 · 60 trivia questions
Is the answer anywhere in the top 5?
SerperGoogle results
3623 first
Firecrawl search
3627 first
Googlescraped via Apify
3322 first
Claude WebSearchlinks only; summary had it 42 times
145 first
out of 60
research/battle/summary_all60.json answer_in_top5, answer_in_top1, claude.answer_in_tool_text · grader eval/grade.py contains_gold
Sixty trivia questions. Exa had the answer somewhere in its top five for forty nine. Tavily put it first most often, tied with Exa at thirty two. The Google family, Serper, Firecrawl and the scraper, sit in the middle.
Claude's tool returns links with no snippet text, so it scores fourteen on links. Its written summary held the answer forty two times. Wikipedia's own search is fast and nearly useless as a snippet source.
3.2 The catch
Our test favours sources that send more text
average characters per result, drawn to scale answer in top 5
Firecrawl search
863
36 / 60
summary_all60.json mean_snippet_chars, answer_in_top5 · answers_summary.json all60.exa.cost_usd, all60.google.cost_usd
Here is the catch. A results page gives you about a hundred and forty characters per link. Exa gives you over fifteen hundred. More text contains more answers, so a "contains the answer" test leans toward long snippets.
Press. And long text is not free: the model's bill for the same sixty answers was three times higher.
Honest reading: Exa and Tavily rank well and send more text. Our test cannot fully separate the two.
3.3 Speed · typical seconds per search (median), 60 trivia
Most sources answer in about 2 seconds. Claude's search takes 16 .
summary_all60.json median_latency_s · results/google.jsonl latency_note (one Apify run per batch) · real time: each dot takes its source's median
Each dot takes exactly its source's median time, in real time. The dedicated APIs all come back in under about two seconds. Scraping is slower. Claude's WebSearch takes about sixteen, because a model turn sits inside the call.
Watch the counters. By the time Claude answers once, Wikipedia has answered about twenty five times.
3.4 Overlap · how many top 5 links two sources share, 60 trivia
Nine sources, but many show the same pages
Google, Serper, Firecrawl Share most links: 0.59 to 0.71 . They seem to use Google's results.
Tavily, partly About 0.4 with the Google group. Similar pages, in its own order.
Exa, Parallel, Linkup, Claude Each returns mostly different pages: 0.22 or less with anyone.
1.0 = the same links. 0 = no links in common. Average over 60 questions. Our reading of the links, not a vendor's statement.
research/battle/summary_all60.json overlap.<a|b>.mean_jaccard (n = 60 per pair)
Each square: for the same question, how many of the top five links two sources share. Bright means the same pages.
Press. Serper, Firecrawl and the Google scraper share most of their links. That looks like Google underneath. Press. Tavily partly overlaps them. Press. Exa, Parallel, Linkup and Claude each live in their own web.
So asking three Google-shaped APIs is asking one source three times. Remember that for the build.
3.5 Round 2 · 15 news questions, 13 to 20 Oct 2026
On last week's news, most sources found the answer
Firecrawl search
137 first
SerperGoogle results
135 first
Googlescraped via Apify
126 first
Claude WebSearchits written summary
10
out of 15 · all sources combined: 15 of 15
research/battle/summary_fresh.json answer_in_top5, answer_in_top1, union_web_answer_in_top5 · questions: picked_fresh.json
Fifteen questions about news from the last seven days. Most dedicated APIs found the answer. Tavily put it first thirteen times out of fifteen. The Google-results APIs found it too, but further down the list.
All sources together found every one of the fifteen. Wikipedia had not caught up.
3.6 Round 3 · only the search results change
On news, search makes the difference
60 trivia questionsthings a model may already know
Memory only, no search 40
Best search: Exa's top 5 45
15 news questions13 to 20 Oct 2026, after the model's training
Best search: Tavily's top 5 15
Search is not for what the model knows. It is for what happened since .
research/battle/answers_summary.json all60 + fresh (none, google, exa, tavily) · model claude-sonnet-5-5, no tools · graded by eval/grade.py correct
Same model, Sonnet, same prompt. Only the evidence changes. Press. On trivia, memory alone got forty of sixty. Google's snippets: also forty. Only Exa's long extracts beat memory, with forty five.
Press. On last week's news, memory got four of fifteen. Every good search source got thirteen to fifteen.
Press. That is the line of the talk.
3.7 Round 3 in full · Sonnet answers from each source's top 5
Search helps only when the model does not already know
What the model read 60 trivia 15 news Nothing (memory only) 40 4 Google (Apify) top 5 40 13 Serper top 5 not run 14 Parallel top 5 38 14 Tavily top 5 38 15 Exa top 5 45 13 Claude WebSearch summary 38 11 Wikipedia top 5 19 / 30 4
"What city is Apple's October 28 product event in?" No search: "Cupertino, California" . With Google or Tavily snippets: "Los Angeles". Correct.
"What percent of Albertans voted to remain in Canada?" No search: "About 70% (best guess, unverified)" . With Google's snippets: "about 76%". Correct.
answers_summary.json (all60, fresh; wikipedia all60 ran on the first 30 only) · answers/fresh/<source>.jsonl
The whole answer round. On trivia, most search sources did not beat memory. On news, every web source beat it by nine to eleven questions.
Press. Two real examples. From memory the model guessed Cupertino. With snippets: Los Angeles, correct. From memory it guessed seventy percent and even said it was guessing. With snippets: seventy six. Correct.
3.8 What went wrong, and it matters
Free sources broke, and our own notes leaked the answers
Free sources with no sign-up broke 28 / 30 DuckDuckGo Lite searches were blocked as bots . OpenAlex refused 8 of 8 : daily free budget used up. Wikipedia refused 20 of 30 ("too many requests") until we added our contact details.
Our library read our own tests 30 / 30 questions: our research library's top results were our own test runs from that morning. Without them: 1 of 30 .
The answer key was out of date 2016 Our key says Tim Hecker's latest album is "Love Streams". A 2016 page matched it and scored as correct . Another result named a newer album, "Shards" (2025).
summary.json ddg, openalex, brain.questions_with_session_transcript_hits · results/retried.jsonl (first failures) · results/google.jsonl hp04
Three things broke, and each one is a lesson. The free sources with no sign-up failed first: blocked as bots, budgets used up, too many requests. Access is part of fairness.
Our own research library had already read this morning's test runs. Every question's top hit was our own homework. And our answer key was stale: an old page matched an old answer and got full marks.
3.9 Which sites come back · 10 questions people argue about, top 5 each
Each source favours different kinds of sites
Linkup
100% everything else: blogs, firms, small sites
Claude WebSearch
94% everything else
Wikipedia
100% encyclopedia
research/battle/contested_summary.json per_source.shares (50 results each) · classes from a hand made domain list in contested.py
Ten questions where sources might slant: rent control, fasting, the best VPN. We sorted every domain into rough classes with a hand made list.
Exa leans to government, universities and papers: thirty percent. The Google family carries about one forum or video result in five. Linkup and Claude mostly returned small sites our list could not place.
About forty percent is "everything else", so read this as a picture, not a measurement.
3.10 What the top 5 looked like
The best source depends on what your agent is for .
"Do rent control policies increase or reduce housing supply?"
Exa ScienceDirect, doi.org, an economics journal site
Parallel the Urban Institute, SSRN, Brookings
Serper Reddit first, then Brookings and a pro-housing advocacy group
Linkup a law firm, a landlord site and a real estate agent
Best VPN every engine: review and affiliate sites. There is no primary source.
Our reading, not a verified finding
"Who invented the telephone?" Claude's WebSearch returned pages on unrelated companies' subdomains:
api-legacy.ingeniusprep.com
These look like spam pages made to rank in search . We did not check further.
research/battle/contested_summary.json examples (rent control, best vpn, telephone) · results_contested/<source>.jsonl
Same question, three personalities. Exa: journals. Serper: Reddit, a think tank, an advocacy group. Linkup: people with a stake in the answer.
Press. Best VPN: everyone returned affiliate sites, because nothing else exists. Press. And one thing we noticed but did not verify: spam-looking subdomains in Claude's results for a simple history question. A source that favours primary pages is better for a research agent and worse for "what do people say".
3.11 Price per 1,000 searches · public pricing pages, read 20 Oct 2026
The most expensive costs 30 times the cheapest. Long text costs more .
SerperGoogle results · 2,500 free searches, once
$1.00
down to $0.50 at 2.5M
Bright Data †5,000 records a month free
$1.50
ran; results held back for a fix on our side
Google via Apify scraper$5 credit a month
$1.80
list price; we paid $4.50 per 1,000
Parallel5,000 requests a month free
$1 to $5
per 1,000 requests, 10 results each
You.com Searchpriced, not tested
$5
Linkupstandard search
$5 to $6
Exa$10 credit a month
$7
Auto · $4 Lite, $12 Deep
Firecrawl searchHobby plan, 2 credits per 10 results
≈ $7.60
Tavily1,000 credits a month free
$8
basic · $16 advanced
Anthropic web search toolplus model costs
$10
SerpAPI$25 a month for 1,000
$25
Claude WebSearch, as we ran itSonnet in Claude Code, model costs included
≈ $30
measured: $1.79 for 60
Google Custom Searchends 1 Jan 2027
$5
not available to new customers
TOS_PRICING.md (quotes each pricing page) · † figure from a search summary; the page renders it in script · results/google_cost.json · summary_all60.json claude.cost_usd_total · bars on a log scale
Public prices, read today (20 Oct). Cheapest pay as you go tier. The spread is thirty times. The SERP resellers are cheapest per query. The AI native ones charge for the longer text they hand back, which, as we saw, is also why they score well.
Two amber rows are what we actually paid. The scraper cost us more than list. Claude's WebSearch through Claude Code came to about thirty dollars per thousand, because a model turn is inside every call.
4
Part 4 · Build it tonight
Ask more than one web
Ask several sources and compare their answers. When they disagree, read the pages.
So what do you build tonight. One pattern, about forty lines: ask two or three sources that see different parts of the web, merge the links, and trust an answer more when independent sources back it.
4.1 The pattern · ask several, merge, compare
Three sources from different webs
search_with_agreement.py async def search_with_agreement(q, sources=(serper, exa, tavily)): got = await gather(*(s.search(q, k=5) for s in sources)) ok = {s.name: r for s, r in zip(sources, got) if r} # log who failed hits = dedupe_by_url(r for rs in ok.values() for r in rs) hits = [h for h in hits if not own_eval(h)] # drop our own test notes answers = [cheap_answer(q, rs) for rs in ok.values()] best, votes = majority(answers) if votes >= 2: return best, cite(hits, best), "agreed" return deep_read(q, top_pages(hits, 3)), "checked"
content/battle.md section L · overlap: summary_all60.json · 72%: vote.json all60.voting (34 of 47 agreed)
Press one: fan out to three sources from different webs. Not three Google-shaped APIs, that is one source three times. Press two: keep who answered and log who failed.
Press three: merge by URL, and drop your own transcripts. We learned that one the hard way. Press four: one cheap answer per source, count agreement.
Press five and six: agreed, answer and cite. Disagreed, go read the pages.
4.2 What three sources gave us · Sonnet answered once from each of Google, Tavily and Exa
When sources agree, the answer is more often right
60 trivia · was the answer right?
Two or three agreed47 questions, 34 right
72%
All three disagreed13 questions, 6 right
46%
On news: 11 agreed, all 11 right . 4 disagreed, 3 right.
Right answers did not go up. The majority answer: 40 of 60 . Same as Google alone, below Exa alone (45).
Merging found almost no new answers. Exa alone 49, all five together 51 . News: 14, then 15 .
Cost. Serper + Exa + Tavily at list price: about $16 per 1,000 questions , before model costs.
More sources do not give more right answers. They tell your agent when to double check .
research/battle/vote.json all60 + fresh: voting, union_answer_in_results · prices: TOS_PRICING.md ($1 + $7 + $8 per 1,000)
Here is what the pattern actually bought. Press. When two or three answers agreed, the answer was right seventy two percent of the time. When all three disagreed, forty six. On news, every agreed answer was right.
Press. The honest part: majority vote did not beat Exa alone. And merging barely added coverage once one strong source was in.
Press. So the pitch is not more right answers. It is knowing when to go read.
4.3 Tonight's tools · public docs
Use Bright Data's MCP for search, and to read full pages
Bright Data MCP
What it gives you
endpoint mcp.brightdata.com/mcp?token=…free tier 5,000 requests a month
default tools search_engine, scrape_as_markdown, discover; 32 in allIn the pattern
Where it plugs in
one more source search_enginethe reader scrape_as_markdown reads the full pageadd it claude mcp add with the endpoint aboveIt ran on all 85 questions. Our code mixed up its replies, so its numbers wait for a fix. Check its overlap before you count it as a new web.
docs.brightdata.com/mcp-server/overview, /tools, /remote/quickstart · held back: research/battle/results/_unreliable/README.md
Rafa just showed you Bright Data's MCP. In this pattern it does two jobs. Its search tool is one more lane in the fan out. Its scrape as Markdown tool is the reader: the step where disagreement sends the agent to the actual page.
It ran on all our questions today (20 Oct), but our own MCP client paired some replies with the wrong questions, so its numbers are held back until we fix that. That is our bug, not theirs. Measure its overlap first, using the same heatmap.
5.1 Honest limits
A snapshot of 20 Oct 2026, not a league table
Small sets. 60 trivia, 15 news, 10 contested. US, English, one afternoon.One search per source , as typed. First 5 results only.Exact text matching. "$12.87 billion" misses a key answer of "$12.9B". An out of date key answer still scores as correct.Text length was not made equal. Each source sent its default amount, which favours long text.We reached Google only through others : an Apify scraper and Serper. Scraper time is split across a batch.Claude's summary may carry the model's own knowledge, not only the search.One model, one prompt in the answer round. Serper answered on the news set only.Overlap is our reading. It shows shared links, not whose index sits underneath.We wrote the news questions on 20 Oct 2026 using web search. Sources that hold those pages start ahead.We sorted sites into groups by hand. About 40% did not fit a group.Bright Data is held back. Our own code mixed up answers; we caught it because a third of results matched no other source.
content/battle.md section M · research/battle/ (every file named in this deck) · vendors change ranking daily
The limits, plainly. Small sets, one afternoon, one query each. Our grader matches strings, so it misses paraphrase and rewards stale answers. We did not equalise snippet length.
Treat every number tonight as a snapshot of 20 Oct, on our questions. Vendors change their ranking every day. Run the same test on your own questions.
Search API Battle Royale Ray Zahid · Immersive Commons
Ask more than one source. When they disagree, have your agent read the page .
agent-search-talks.pages.dev/battle
One line to take home. Search is for what happened since the model was trained, and no single source sees all of it. Ask two or three webs, and when they disagree, read the page.
The deck is at agent-search-talks.pages.dev slash battle; all four talks are on that site. Thank you.
short cut 01 / 01
Overview · arrows + Enter or click · Esc to close