Crawling and Indexing Explained
Being crawled, being indexed, and ranking are three separate things. Most "we don't rank" problems are actually a failure at one of the first two.
Intermediate8 min readDaksh BathlaDEFINITIONSearch & AI Discovery
Crawl Budget
The rough number of pages a search engine will fetch from your site in a period, based on how important the site seems and how fast it responds.
EXAMPLE
A shop with filter and sort parameters generating fifty thousand near-identical URLs spends its budget on those instead of on product pages.
WHY IT MATTERS
Above a few thousand URLs it decides which of your pages get seen. Below that it almost never matters — and optimising for it on a small site is a way of avoiding work on content.