What is crawl budget, and does it matter for my site?
The amount of crawling a search engine is willing and able to do on a site — and for the overwhelming majority of sites, it does not matter at all, which is the most useful thing to know about it.
What determines it. Google describes two components:
Crawl rate limit — how fast the crawler can fetch pages without degrading your server. If your site responds slowly or returns errors, crawling is reduced to avoid harming it.
Crawl demand — how much the crawler wants to fetch, based on popularity and how often content changes. A frequently updated, frequently linked site attracts more crawling.
Who it actually affects. Google's own guidance is explicit that crawl budget is not a concern for most sites. It becomes relevant for:
Very large sites — in the region of hundreds of thousands or millions of URLs.
Sites generating enormous numbers of URLs automatically, typically through faceted navigation where filter combinations multiply into effectively infinite URLs.
Sites with frequently changing content where recrawl speed matters commercially.
Sites with serious technical problems producing slow responses or large volumes of errors.
If your site has a few thousand pages, crawl budget is not your problem, and time spent on it is time not spent on content or links.
What wastes it where it does matter:
Faceted navigation generating crawlable combinations of filters — the single largest cause.
Infinite spaces such as calendars with no end date.
Duplicate URLs from parameters and session identifiers.
Soft 404s, where missing pages return a success status.
Long redirect chains.
Slow server responses, which reduce the rate directly.
Large volumes of low-value pages — thin tag archives, empty search result pages.
What helps: blocking genuinely worthless URL patterns in robots.txt; returning proper status codes; improving server response time; keeping sitemaps accurate and current; and consolidating or removing low-value pages.
The distinction worth keeping clear: crawling is not indexing. A crawled page may not be indexed, and "crawled — currently not indexed" usually indicates a quality judgement rather than a budget problem.