Build Local Pages AI Systems Can Actually Find

TL;DR: AI systems cannot surface a local business page they cannot access, render, or trust. Cloudflare blocking, JavaScript-heavy builds, and stale orphaned URLs are killing visibility before any content optimization matters. Fix crawl access and data accuracy first, then optimize copy.

Crawl Access Is the Starting Point, Not an Assumption

Whitespark founder Darren Shaw put it plainly: “You cannot be surfaced in AI responses if the AI can’t even access your website.” That sentence should precede every local page audit conversation. Before you rewrite service copy or add FAQ schema, confirm that crawlers can physically reach the page.

Russ Jeffery, Director of Platform and Product Strategy at Duda, listed the basic controls: robots.txt directives, noindex tags, security settings, sitemaps, and rendering method. He admitted to launching a site with a noindex tag accidentally left in place. His conclusion: fixing it later “takes a heck of a lot longer than just doing it right the first time.”

Shaw flagged Cloudflare as a silent blocker that many operators never know is active. “Cloudflare often blocks AI crawlers by default,” he said, and the setting is frequently enabled by a web developer or host without informing the business. He cited a Shopify site where Cloudflare was the sole culprit. His fix: check whether Cloudflare is running on the domain, then audit its crawler-blocking settings explicitly.

Both Shaw and Jeffery pointed out that the default block-all policy was designed for major publishers like The New York Times, whose content has licensing value. “But every small business in the world, they do not want to block crawlers.” Jeffery called it “a bad default.” For operators running high-spend paid acquisition, losing AI organic visibility to an invisible infrastructure setting erodes the full-funnel return on every dollar spent. A thorough marketing audit should include a crawl-access check as a line item, not an afterthought.

Server-Side Rendering Is Now a Visibility Requirement

Jeffery described JavaScript rendering as having “gone backwards in the past few years” relative to AI crawler capabilities. Google still executes JavaScript and handles it reasonably well. Newer AI systems largely do not. ChatGPT, for example, does not maintain its own index and does not take time to render client-side JavaScript during discovery.

His guidance: deliver all important content in the raw HTML response through server-side rendering, then verify what crawlers actually receive. Do not assume your framework handled it. WordPress, Next.js, Duda, and Wix all serve rendered HTML by default. The risk sits with newly generated sites built heavily on client-side React, particularly those created with tools like Claude Code or similar AI coding assistants.

Shaw was direct about vibe-coded sites: “If you’re just vibe coding a website, they’re usually pretty bad, and I would just pay attention to that.” His reassurance for established platform users: the vast majority of small business sites do not have this problem, but operators should verify that specific dynamic elements like review carousels or pricing widgets are not loading exclusively via JavaScript after page render.

Jeffery also mentioned markdown delivery as an emerging option Cloudflare is promoting. His verdict: not required yet. No major AI search engine currently relies on a markdown version of a page. Accessible, server-rendered HTML remains the priority.

Orphaned Pages Feed AI Systems Wrong Data

Crawl access becomes a liability when the crawlable content is outdated. One of the most common problems in local page audits is orphaned or duplicated URLs, pages left behind during site redesigns with labels like “-old,” “-new,” “/home,” or “v2.” These pages often carry stale phone numbers, old addresses, or discontinued service areas.

Customers rarely land on them through navigation. Crawlers do. As Shaw put it, “the AIs would grab it.” A technical audit focused only on broken links misses these entirely. The audit also has to check for indexable stale versions. The fix is straightforward: run a full crawl, filter for URLs carrying those suffixes, confirm none of them are indexable, and de-index or canonicalize any that are.

This problem compounds quickly for operators running multi-location campaigns. A law firm running mass tort acquisition across fifteen states, or a licensed iGaming operator with state-specific landing pages, can have dozens of orphaned URLs feeding AI systems the wrong jurisdictional information. The crawl audit must be scope-aware, not just a single-pass check.

Schema Is a Maintenance Obligation, Not a One-Time Win

Schema was the one topic where Shaw and Jeffery diverged, and the disagreement is operationally useful. Shaw is skeptical: “I’ve never seen any noteworthy study that said, if you do schema, your traditional rankings will go up or your AI visibility will go up.” He cited detailed testing by Jake Hundley that found no measurable effect.

Where Shaw does see value is disambiguation. Structured data makes product or service information unambiguous in a way that page layout cannot match. That is useful for parsers, even if it does not lift rankings directly.

Jeffery’s position: implement it, but treat it as a third live source of business data alongside the website and the Google Business Profile. The worst-case scenario is schema that goes stale. He described clients who update a phone number on the visible page but do not update the schema markup, leaving the old number active in the structured data. Shaw flagged the same pattern: a site built in 2017 where the developer added schema, the owner rebuilt the site in 2025, and no one touched the RankMath schema setting because they did not know it existed.

The practical rule Shaw endorsed: map every data point in your Google Business Profile to schema, and add schema updates to the same checklist as any address, phone number, or service area change. Treat it as a business record, not a technical decoration. Operators scaling across verticals with managed performance ads that feed into local landing pages need a documented sync process between paid campaign assets, live page copy, and schema markup or stale data will undermine the AI visibility those campaigns depend on.

What an AI-Ready Local Service Page Actually Contains

Shaw’s research process for building an AI-ready local page starts with asking the AI itself. His working example was a plumber’s hot water tank repair page, already optimized for traditional SEO, now getting a second pass for AI visibility. He queries Google’s Ask Maps (Gemini grounded in Maps data), Gemini, or Claude directly: what should be on this page? The answers are consistent: pricing, trust signals, reviews, and case studies. Jeffery added credentials and service area specifics.

Step two is query fan-out, using Mark Williams-Cook’s queryfan.com. Shaw’s illustration: a user tells a chatbot their hot water tank failed last night, they need it repaired fast, and their budget is tight. To answer that single prompt, the AI runs approximately ten internal searches. Those ten searches become the page’s FAQ list. The more of those sub-queries a page answers, the higher the probability it gets cited when the original prompt comes in.

Jeffery added a low-tech source: ask the business owner what customers ask most often. Real questions from real customers are more useful than any keyword tool because they reflect intent at exactly the moment a prospect is making a decision. For crypto acquisition campaigns or forex lead generation landing pages, that means documenting objections from sales calls and support tickets, then building them into the service page structure as explicit answers, not generic copy.

Operators running precision-targeted acquisition into geo-specific landing pages need to apply this same framework at the location level. Each city or state page should answer the sub-queries that a local user’s prompt would generate, not simply swap a city name into a template and call it localized.

What This Means for Performance Marketing Operators

The technical foundations covered in this session apply directly to operators running paid campaigns into local or geo-targeted landing pages. Every dollar spent driving traffic to a page that AI systems cannot access or trust is buying visibility in only one channel. As AI-influenced search grows as a discovery layer, pages that fail the crawl and rendering tests will lose attribution from organic AI citations while still incurring full paid acquisition costs.

The operational checklist from this session is short and executable. Confirm Cloudflare is not silently blocking AI crawlers. Verify that critical page content is present in the raw HTML response, not loaded by JavaScript after render. Run a crawl audit filtered for orphaned or stale URL variants and de-index them. Treat schema as a live data record synced to every business detail update. Build FAQ content from actual query fan-out research and real customer questions, not keyword volume reports.

For operators who have not run a structured crawl-and-content audit recently, that is the correct starting point before any copy or schema work begins. The session’s core principle, restated: optimization work cannot help an AI response if the system cannot access the source.

Originally reported by Search Engine Journal, September 2026.