White label AI search optimization for WordPress sites
The technical side of AI search for your clients’ sites, done by developers and handed over under your agency’s name.
ChatGPT, Perplexity, Claude and Google’s AI Overviews can only quote a page they can reach and read. On a lot of WordPress sites they can’t. A Cloudflare setting blocks the bots, the important text loads with JavaScript, or the schema says almost nothing about the business. Our AI search optimization for WordPress starts with a free audit that finds those problems. Then we fix them in the theme, the plugins and the server setup.
Why AI search skips good client sites
-
AI search bots are blocked, and nobody remembers doing it
-
Key text only appears after JavaScript runs
-
Schema says “WebPage” and little else
-
Half the site isn’t in Bing at all
-
Service pages never answer a question directly
-
Slow pages time out for crawlers on a budget
Most of this is invisible from the front end. The site looks fine to people and still gives AI search very little to work with.
What AI search needs from a WordPress site
There is no secret ranking factor for AI answers, and anyone selling one is guessing. What we can say for sure is how the systems get their material. Each one has a crawler or a search index behind it, and each one prefers pages that are easy to fetch, easy to parse and easy to quote.
Access
OpenAI’s OAI-SearchBot, Claude-SearchBot and PerplexityBot have to be allowed in robots.txt and at the firewall. Google’s AI Overviews and AI Mode use the normal Google index, so Googlebot matters there, not a separate AI bot. Microsoft Copilot answers come from Bing.
Readable HTML
AI crawlers fetch the page and read what the server sends. Anything a script fills in later may never be seen. In our tests that meant animated counters, the kind that hold a site’s “17+ years” claim. Review widgets and client-side apps load the same way by design. Tabs and accordions in Elementor, Kadence and Spectra were fine.
Clear facts about the business
Who the company is, what it offers, where and for whom. Organization, LocalBusiness, Service, Article and FAQPage schema spell that out in a way machines don’t have to guess at, and they connect the pages into one picture of the business.
Pages that answer
AI answers quote short, specific passages. A service page that opens with two paragraphs of brand story and hides the actual answer in an accordion halfway down gets skipped for one that says it in the first lines.
What our AI search optimization for WordPress covers
This is developer work, done on staging first. We don’t write your client’s content or run their SEO strategy. We make sure the site lets the content do its job.
Crawler access
robots.txt (the physical file, Yoast’s editor or the robots_txt filter, whichever the site uses), Cloudflare AI Crawl Control and WAF rules, security plugins that block unknown user agents, and host-level bot protection on WP Engine, Kinsta or Cloudways. Search bots get in. Training bots stay blocked if your client wants that.
Server-rendered content
Counters, “load more” lists and widgets that render empty get fixed so the text is in the HTML from the start. We tested Elementor, Kadence and Spectra: tabs pass, counters don’t, and a small filter fixes them.
Schema and entity data
We extend the Yoast or Rank Math graph instead of pasting JSON-LD into the footer. Organization with sameAs links, Service pages, authors with real bios, FAQPage where the questions are real. Then we validate it and fix the warnings.
Page structure
One H1, honest H2s, a short answer near the top of each key page, tables where a comparison is buried in prose. We send your writers a list of pages that need a content change, and they decide what to do with it.
Indexing in Google and Bing
Sitemaps that list the right URLs, noindex left over from staging, canonical tags that point to the wrong place, IndexNow for Bing, and the Bing Webmaster Tools account that most sites never set up.
Speed for bots and people
Server response time, caching rules and heavy pages. Crawlers give up on slow responses, and slow sites also lose the classic rankings that AI search still leans on. Deep speed work goes through our WordPress speed optimization service.
What’s in the free AI readiness audit
Send us the URL. You get a short written report you can forward to your client as it is, with your logo or without any branding. The audit is free. The fixes are quoted in hours after it, and pricing is on request.
- Which AI and search bots can reach the site, tested from outside, not just read from robots.txt
- Cloudflare, host and security plugin settings that block or challenge bots
- Five key pages compared as a browser sees them and as a crawler gets them
- What the schema says about the business, and what it’s missing
- Google and Bing indexing gaps: sitemaps, noindex, canonicals
- Real and fake AI bot visits in the server logs, checked against the vendors’ IP lists
- A list of fixes ordered by impact, with an estimate in hours
We ran the audit on our own site first
Real numbers from deviart.agency, October 17 to November 16, 2026. The site runs on Cloudways behind Cloudflare. We checked every bot request against the IP lists the vendors publish. Two findings surprised us: almost a third of the “AI bot” traffic was fake, and Perplexity’s real crawler never reached the server at all. The full log analysis, with the script, is in our guide to AI crawlers and robots.txt on WordPress.
| What we found | What we did | |
|---|---|---|
| AI search bots in robots.txt | All allowed | Nothing to change |
| Real OAI-SearchBot visits | 99 in 31 days, every one answered with 200 or a redirect | Nothing to change |
| Real Perplexity visits | 0 in 31 days, while 81 fakes used its name | Checking Cloudflare’s rules for Perplexity |
| Fake “AI bots” | 113 of 350 requests using OpenAI’s bot names came from other IPs, mostly scanners after .env and key files | Filtered out of our reports |
| Page text in the raw HTML | Same word count with and without JavaScript | Nothing to change |
| llms.txt | Generated by Yoast, no service pages in it, never requested by an AI crawler | Kept, low priority |
| Bytespider (ByteDance) | 403 from the server, a block we never set up | Left blocked, it’s a training crawler |
What we don’t promise
AI search is new and it changes every few months. We’d rather tell you where the limits are than sell a guarantee nobody can keep.
No guaranteed citations
Nobody can promise that ChatGPT or Google will quote a site. We remove the technical reasons they can’t. Whether they do also depends on the content, the brand’s reputation and the competition.
No magic file
llms.txt is a proposal, not a standard. Ours was requested 78 times in a month, by Lighthouse, SEO tools and people, and not once by an AI crawler. We’ll add one if you want it. We just won’t pretend it moves anything.
No content farm
We don’t generate pages of AI-written text to “feed the bots”. That tends to hurt a site in both classic and AI search. The writing stays with your team or your client.
How we work with your SEO and content team
Most agencies that sell AI visibility already have an SEO lead. We fit in under them.
You own the strategy
Which questions to target, which pages to write, how to track mentions of the brand. We don’t touch that unless you ask.
We own the site
Access, rendering, schema, indexing and speed. Changes go to staging, your team checks them, then we deploy.
One report, your brand
Before and after for every fix: what the bots could reach, what changed in the HTML and the schema. Your client sees your agency’s name, not ours.
How it works
-
1. Send the site
The URL is enough for the first look. Read-only access to Cloudflare, the host and Google Search Console makes the audit more precise.
-
2. Free audit
Within 2 working days you get the report and an estimate in hours. Most sites need between 8 and 25 hours of work.
-
3. Fix on staging
We make the changes on a staging copy, compare key pages with the live site and check that forms, tracking and checkout still work.
-
4. Deploy and recheck
We go live, test bot access again from outside, and send the report. A month later we check the logs and Bing Webmaster Tools to see which bots came back.
Ways to order
The audit is free. Pricing for the work is on request, after the audit.
AI search optimization FAQ
It overlaps a lot. AI answers draw on search indexes, so a site that is indexed, fast and clearly structured has a head start. The differences are extra crawlers to let in, content that has to be readable without JavaScript, and schema that describes the business well. We do the technical part. Your SEO team keeps the strategy.
Block the training bots if your client wants that: GPTBot, ClaudeBot, Google-Extended. Keep the search bots open: OAI-SearchBot, Claude-SearchBot, PerplexityBot. Blocking those removes the site from those products’ answers. Each one is a separate line in robots.txt, so your client can choose.
Not for Google, which says its AI features don’t need any special files. In a month of our own logs, no AI crawler requested our llms.txt even once. It doesn’t hurt, and Yoast can generate one, so we add it if you ask, but we don’t sell it as a fix.
Three ways. Server logs show which AI bots fetch which pages, once you filter out the fakes: on our own site almost a third of the requests using OpenAI’s bot names came from other IPs. Analytics shows visits from chatgpt.com, perplexity.ai, claude.ai and gemini.google.com. Google Search Console counts AI Overviews and AI Mode traffic inside the normal web search numbers. We set up a GA4 channel for AI referrals so you can report it to your client.
Yes. We tested Elementor, Kadence and Spectra: tabs, accordions and sliders keep their text in the HTML, but counters only show their numbers after JavaScript runs, so those need a fix. For WooCommerce we make sure product data and Product schema are in the HTML and that the store’s filters don’t waste the crawl.
Related services
Related reading
Is a client asking why they don’t show up in ChatGPT?
Send us the site. In 2 working days you’ll know what AI search can and can’t see, and what it takes to fix.