CDN migration SEO: Google's crawl-spike warning

Moving your site behind a CDN can briefly increase the load on your origin server, not reduce it. Google explained why on 11 November: faster responses tell Googlebot it can crawl harder, and a brand-new CDN cache is empty. Separately, blanket bot protection on the CDN can lock Googlebot out, and if that goes on for long enough, pages and Shopping listings start dropping out of Search.

If you are planning a move to Cloudflare, Akamai, CloudFront or any other CDN before the year-end and Republic Day sale season, read this before you flip the DNS.

What Google said

The post, by Cherry Prommawin and Gary Illyes of Google Search Central, accompanies the last episode of Google's Non-Canonical Cabinet video series. It makes three points.

1. Faster sites get crawled harder

Google sets how hard it crawls a host by watching how fast and how reliably the server responds. A CDN usually cuts response times in one step, so Google's systems read that as a healthier server and gradually raise the crawl rate. Crawl demand still matters: a faster site raises the ceiling, but Google will not fetch URLs it has no reason to want.

2. An empty cache sends crawls back to your origin

The first request for each URL at each edge location is a cache miss, and that miss goes back to your origin, whether it is HTML, CSS, JavaScript or an image. Shoppers cluster on your homepage and a few best-selling product pages, so those warm up quickly. Googlebot walks the whole URL inventory: old category filters, product variants, blog archives. Most of those requests miss the cache. Google's line sums it up: "Caches only shield you on the second visit."

3. Blanket bot rules block Googlebot too

CAPTCHAs, JavaScript challenges and "are you human" interstitials that a CDN or WAF applies to all automated traffic treat Googlebot like any scraper. Googlebot fails the challenge, gets a page with nothing useful on it, and backs off. Google warns that if the block stays, pages and shopping listings begin to fall out of Search. A 403 tells Google access is refused, so Google advises against 403 walls and challenge pages for crawlers.

What it means for Indian sites

This matters most for large catalogues. A D2C fashion brand with 4,000 SKUs, colour and size variants and filter URLs can easily expose 50,000+ crawlable URLs, while real shoppers touch a few hundred. On a ₹3,000-a-month VPS or a modest shared WooCommerce host, a week of Googlebot cache misses can push the origin into timeouts, and timeouts cut your crawl rate right when you want new sale pages indexed.

The bot-blocking risk is the bigger one. We regularly see "Under Attack" or bot-fight modes switched on site-wide during a traffic spike or a scraping incident and then forgotten. Ranking drops that follow often get blamed on a core update. With the November 2026 core update rolling out now, rule out a crawl block before you read anything into a drop.

How to move to a CDN without losing crawl

Google's own recommendations, in order:

• Allow verified bots. If you need bot filtering, turn on your CDN or WAF's verified-bot option or managed rule rather than challenging everything. Verify Google's crawlers against Google's published IP ranges or reverse DNS, not by user-agent, which anyone can fake.
• Target the bad bots, not all bots. Use robots.txt or specific rules for scrapers you actually want gone.
• Pre-warm the cache. Before sending all traffic through the CDN, fetch your high-traffic pages, upcoming sale and campaign landing pages, and your most-crawled URLs at a controlled pace.
• Throttle briefly if the origin is struggling. Return 429 or 503 for a short spell, which Google puts at "a few hours, or 1-2 days at most". Its guide to reducing crawl rate covers this. Longer than that and you risk URLs dropping from the index.

Our opinion: do the switch in a quiet week, not the week your sale goes live. If you can only migrate in sale week, pre-warm the sale URLs a day early and keep someone watching origin CPU for the first 72 hours.

Before, during and after the switch

If you are also cleaning up thin pages around the same time, don't mix that with the CDN move. Content pruning won't fix indexing on its own, and doing both at once makes it impossible to tell which change caused what.

What to do this week

• Open Crawl stats in Search Console and screenshot the last 90 days as a baseline.
• Log in to your CDN or WAF and check whether any site-wide challenge, "bot fight" or "under attack" mode is on. If it is, switch to a verified-bot allow rule.
• Run URL Inspection's live test on your homepage, one category page, one product page and one blog post. Check the rendered HTML is your page, not a challenge screen.
• If a migration is planned, write the pre-warm URL list now and schedule the move away from sale launches.
• Make sure your team knows 429/503 is a brake for hours, not a setting to leave on.

FAQ

Does moving to a CDN hurt SEO?

Not by itself. Google's post says faster responses can lead to more crawling, which is usually good. The risks are temporary origin overload from cache misses and Googlebot being blocked by bot protection.

How do I know if my CDN is blocking Googlebot?

Use URL Inspection's live test in Search Console and look at the rendered page and HTTP response. Check your CDN firewall logs for challenges or 403s served to Google's verified IP ranges, and watch Crawl stats for a sudden fall in requests.

How long can I return 503 to slow Googlebot down?

Google suggests a few hours, and one to two days at most. Use it only when the origin is under real strain.