Your Agent Readiness Score Is Probably Wrong

TL;DR: Cloudflare’s isitagentready.com scanner scores websites on AI agent readiness across five categories and 16 checks — but the default preset inflates failure rates for content sites, the composite number is unreliable until you select the right site-type configuration, and several scored checks reflect emerging proposals with near-zero real-world adoption. Fix what maps to live agent runtimes. Watch the rest.

What Cloudflare Actually Shipped

On April 17, 2026, Cloudflare launched isitagentready.com — a free public scanner that takes any URL and returns a scored report on how legible that site is to AI agents. Paste a URL, pick a website type, hit Scan. The tool fetches the homepage and a set of well-known paths, runs 16 checks, and outputs a composite score with pass/fail markers, HTTP status codes, and AI-generated remediation guidance.

The scanner is also available through Cloudflare Radar, via the URL Scanner API for programmatic automation, and — notably — as a stateless MCP server endpoint. Any MCP-compatible agent can call the scanner as a tool and reason over the result before deciding how to interact with your site. The measurement infrastructure and the agents being measured now share the same surface. That is not a minor detail.

The broader context: the scanner arrived alongside Agent Memory, Shared Dictionaries, Redirects for AI Training, an LLM compression technique called Unweight, and a feature-flag tool called Flagship for AI-generated code — all shipped during Cloudflare’s Agents Week. The readiness scorer is the logical closer: if agent runtimes are the new browser layer, site owners need a way to test whether their site is legible to that layer. Cloudflare shipped the tester.

16 Checks, 5 Categories: What Gets Scored

The scanner groups its checks into five categories. Understanding what each one actually tests determines which failures are worth acting on immediately and which are watch-list items.

Discoverability (3 checks): robots.txt exists, sitemap.xml exists, and HTTP Link headers per RFC 8288. The first two are table stakes — any agent that follows crawl policy needs them. Link headers are useful for agents that parse HTTP responses rather than rendered HTML.

Content (1 check): Markdown content negotiation. The scanner sends an Accept: text/markdown header and checks whether the site returns Markdown instead of HTML. Real agent runtimes prefer Markdown because it is cheaper to tokenize and easier to parse. Most sites do not support this yet, but it maps to live runtime behavior today.

Bot Access Control (3 checks): AI-specific rules in robots.txt (GPTBot, ClaudeBot, PerplexityBot, etc.), Content Signals directives in robots.txt (an emerging spec with minimal adoption), and Web Bot Auth request signing at a well-known path (adoption is near zero outside Cloudflare’s own properties).

API, Auth, MCP and Skill Discovery (6 checks): API Catalog per RFC 9727, OAuth/OIDC discovery, OAuth Protected Resource metadata, MCP Server Card, Agent Skills index, and WebMCP in-page JavaScript tools. These six checks account for a substantial share of the composite score — and all six will fail on any standard content site.

Commerce (3 optional checks, not scored on non-commerce sites): x402 payment protocol, UCP profile, ACP discovery document. Cloudflare correctly excludes this category from the composite score if no e-commerce signals are detected. That design decision matters and signals the team anticipated the scoring bias problem described below.

The Default Preset Is Wrong for Most Sites

The same site can score 33 out of 100 under the default All Checks preset and 67 out of 100 under the Content Site preset — same scanner, same site, same day. That 34-point gap reflects two different scan configurations, not two different levels of actual agent-readiness.

The Content Site preset removes every check in the API/Auth/MCP/Skill Discovery category, every Commerce check, and Web Bot Auth from Bot Access Control. Six scored checks remain: three Discoverability checks, one Content check, two Bot Access Control checks. Those six map directly to standards that agent runtimes act on today.

The problem is that the preset is hidden. When a user lands on isitagentready.com and pastes a URL, the default scan runs All Checks. The Site Type toggle lives inside a Customize dropdown that most users never open. The shareable number — the one that circulates on social media and gets compared across competitors — is the All Checks composite. For a content site, that number is structurally too low. The accurate reading is behind a dropdown.

Any operator running this scanner on their own properties before briefing a team or sharing results publicly needs to open Customize, select the preset matching the site type, and re-run. Without that step, the reported score understates actual agent-readiness, and the gap is larger for content sites than for API-first applications.

What This Means for Performance Marketing Operators

For operators running paid acquisition across high-CAC verticals — forex lead generation, iGaming media buying, law firm intake campaigns, crypto exchange marketing — the Agent Readiness Score surfaces a class of technical debt that has nothing to do with ad copy or bid strategy. It measures whether your landing pages, lead forms, and content assets are structurally legible to AI-driven crawlers and agents.

That matters now for two reasons. First, AI-powered search products (Perplexity, ChatGPT Search, Google AI Overviews) already influence the top-of-funnel queries your paid media is targeting. A landing page that agents cannot parse cleanly is a landing page that earns fewer citations and lower relevance signals in those products. Second, as AI agents for lead qualification become a standard part of operator stacks, the same delivery infrastructure that makes a site agent-legible for search is the same infrastructure those qualification agents interact with on the way through your funnel.

The practical playbook is straightforward. Start with a full marketing audit that includes a scan under the correct Content Site or API/Application preset — not the default All Checks composite. Fix the checks that map to live runtime behavior: robots.txt AI bot rules, sitemap currency, and Markdown content negotiation if your stack supports it. Add schema.org JSON-LD structured data for your primary content types — this is not a scored check, but it is the highest-leverage fix for citation behavior across every deployed agent runtime. Treat the proposal-stage formats (MCP Server Card, Agent Skills, WebMCP, llms.txt) as a watch list, not an immediate build queue.

Operators running performance ad campaigns at scale should also understand the distinction between content-layer optimization (SEO, CRO) and transport-layer optimization (agent-readiness). You can improve an agent-readiness score without rewriting a single word of copy. You can also have world-class SEO content that scores a 10 on the agent-readiness scanner because the delivery pipeline was built for human browsers, not machine consumers. These are adjacent disciplines occupying different layers of the stack, and conflating them produces the wrong remediation plan.

For CDL recruitment operators running high-volume landing pages and job listing assets, the most actionable near-term fix is the same: clean robots.txt with explicit AI bot directives, a current sitemap, and structured data markup on job posting content. Those three steps cost less than a half-day sprint and address every check that maps to agent runtime behavior available today.

Three Goodhart Risks the Score Creates

Goodhart’s Law: when a measure becomes a target, it stops being a good measure. The Agent Readiness Score is now public, shareable, and compared across competitors — which produces three predictable failure modes.

First, site owners will optimize for the number rather than real agent behavior. Add an MCP Server Card that points nowhere. Publish an Agent Skills index with no actual skills. Ship a WebMCP tool that does nothing. The composite score rises, and nothing changes for actual agent runtimes visiting the site.

Second, consultancies will start selling “Agent Readiness Score optimization” as a packaged service. The history of SEO is instructive: PageRank became a target and a decade of link-spam economy followed. Core Web Vitals became a target and a generation of performance-theater optimizations followed. The same gravity applies here. The scanner’s per-check detail and honest category exclusions are worth more than any composite number a vendor pitches against.

Third, scoring emerging proposals as if they were ratified standards accelerates adoption past the point where those proposals are ready to carry real traffic. Several scored checks (llms.txt, Content Signals, Web Bot Auth, WebMCP) have no governing body, competing structural proposals, and near-zero deployment outside Cloudflare-aligned crawlers. A site owner fixing a failing check is the marginal adopter who tips a proposal into de facto standard status before the spec work is complete.

The fix: track individual check outcomes over time, not the composite. A site that moves from 3 of 5 real-runtime checks passing to 5 of 5 has measurably improved even if the composite barely moved. Use the precision audit approach — isolate the signals that matter, ignore the ones that don’t, and re-scan after every meaningful change.

The Six Fixes Worth Acting on Now

Ordered by leverage against live agent runtime behavior, not composite score impact:

1. Run the scan with the correct preset. All Checks is the wrong default for content sites. Open Customize, select Content Site or API/Application, and re-scan before acting on any number.

2. Fix robots.txt AI bot rules. If you have no robots.txt, add one. If you have one, add explicit directives for GPTBot, ClaudeBot, PerplexityBot, and Google-Extended. This is a single-file edit and passes a real crawl-policy check that every major AI crawler currently reads.

3. Verify sitemap.xml currency. Generate one if absent, link it from robots.txt, and confirm it reflects your current URL structure. Agent runtimes that enumerate pages use it.

4. Ship schema.org JSON-LD structured data. Not a scored check, but the highest-leverage fix for citation behavior across every deployed agent runtime. Article, Product, Organization, BreadcrumbList — pick the types relevant to your content.

5. Configure Markdown content negotiation if your stack supports it. Respond to Accept: text/markdown with a Markdown version of page content. Cloudflare’s AI Crawl Control has native support. Other stacks require custom server logic. This is a real delivery optimization for agent parsing efficiency today.

6. Treat MCP Server Card, Agent Skills, Web Bot Auth, and WebMCP as a watch list. Monitor which of these show up in every scanner that ships over the next six months. The checks that appear across multiple vendor scanners are the de facto standards. The checks that only appear in Cloudflare’s scanner are Cloudflare’s bets. Some will win.

The composite score is the marketing layer. The per-check report is the signal. Run the scan, read the report, fix what maps to live agent behavior, and watch the rest.

Originally reported by Search Engine Journal, May 2026.