# 429 Too Many Requests: rate limiting and crawlers

> 429 Too Many Requests means a client hit a rate limit; Googlebot treats it like a server error and slows its crawling.

URL: https://yes-seo.com/http-status-codes/429

**429 Too Many Requests** means a client has sent more requests than the server allows in a given time: it has hit a rate limit. It’s how APIs, CDNs and security tools push back against overload, scraping and brute-force attempts, and it’s the one 4xx code Googlebot treats as a reason to slow down rather than as a sign that a page doesn’t exist.

## Definition

429 isn’t in the core HTTP spec; it was added by [RFC 6585, section 4](https://www.rfc-editor.org/rfc/rfc6585#section-4) in 2012. The response should explain the condition and may include a `Retry-After` header saying how long to wait, in seconds or as a date. The RFC deliberately doesn’t define how a server identifies the client or counts requests (per IP, per account, per API key), and it says 429 responses must not be stored by caches.

## How Googlebot handles 429

[Google’s documentation on HTTP status codes](https://developers.google.com/search/docs/crawling-indexing/http-network-errors) says Googlebot treats 429 as a signal that the server is overloaded, the same way it treats a server error. Crawling slows down. Indexed URLs stay in the index for a while, but URLs that keep returning 429 are eventually dropped. If `robots.txt` itself returns 429, Google temporarily treats the whole site as disallowed.

That makes 429 a legitimate emergency brake. Google’s guide to [reducing the crawl rate](https://developers.google.com/search/docs/crawling-indexing/reduce-crawl-rate) suggests returning 500, 503 or 429 when Googlebot is overloading your server, but only for a short period, on the order of hours or a day or two. Google retired the Search Console crawl rate limiter in January 2024, so status codes are now the main lever.

## Accidental 429s

The more common problem is a rate limit that catches crawlers you want. Typical sources:

- CDN or WAF rate-limiting rules with low thresholds.

- Security plugins that throttle any client making many requests.

- Limits keyed on IP behind a proxy or CDN, where all traffic seems to come from a handful of addresses.

- Hosting plans with request caps.

## How to fix it

### For site owners

- Check Search Console’s Crawl Stats report and your CDN or firewall logs for 429s served to Googlebot or Bingbot.

- Raise thresholds or exempt verified search engine crawlers; verify them by reverse DNS or published IP ranges, not by user-agent.

- Don’t use 429 as a long-term block. To keep a bot out, use [robots.txt](https://yes-seo.com/glossary/robots-txt) or a [403](https://yes-seo.com/http-status-codes/403).

### For developers

Nginx’s `limit_req` answers with 503 by default; set it to 429 so clients know it’s a limit, not an outage:

```
limit_req_zone $binary_remote_addr zone=perip:10m rate=10r/s;

server {
    location / {
        limit_req zone=perip burst=20 nodelay;
        limit_req_status 429;
    }
}
```

When your own code calls an API that returns 429, back off: honor `Retry-After` if it’s present, and otherwise retry with growing delays plus some random jitter.

## How to check

The [HTTP status checker](https://yes-seo.com/http-status-checker) shows whether URLs return 429 to an outside client, and the [HTTP header checker](https://yes-seo.com/http-header-checker) reveals the `Retry-After` value. Compare [503](https://yes-seo.com/http-status-codes/503), the code for a server that’s unavailable to everyone.

```
HTTP/1.1 429 Too Many Requests
Content-Type: application/json
Retry-After: 120

{"error":"Rate limit exceeded. Try again in 2 minutes."}
```

## Frequently asked questions

### Does a 429 error hurt SEO?

Briefly, it only slows Googlebot down. If URLs keep returning 429 over a longer period, Google eventually drops them from the index, so exempt verified crawlers from aggressive limits.

### How do I stop Googlebot from hitting my rate limits?

Allow verified Googlebot IP addresses through your rate-limiting rules, or raise the thresholds. Googlebot already adjusts its crawl rate to how quickly and reliably your server responds.

### Should I use 429 or 503 when my server is overloaded?

Use 429 when a specific client exceeds its quota and 503 when the whole service is temporarily unavailable. Google treats both as signals to slow down.