401 Unauthorized: authentication required

401 Unauthorized means the request needs valid authentication and didn’t have it. Despite the name, it’s about authentication (who you are), not permission: send working credentials and the server may let you in. When the server knows who you are and still refuses, that’s 403 Forbidden.

What the spec requires

RFC 9110, section 15.5.2 says a 401 response must include a WWW-Authenticate header with at least one challenge, telling the client how to authenticate, for example Basic or Bearer. If the request already carried credentials, the 401 means they were refused. Browsers react to a Basic challenge by showing their built-in login prompt.

Common causes

• HTTP Basic authentication on a staging site, admin area or directory (.htpasswd).
• Expired, missing or malformed API tokens (Authorization: Bearer …).
• Protection left on after launch, or applied to a whole domain instead of one folder.
• Protected CSS, JavaScript or image paths, which break page rendering for crawlers even when the HTML is public.
• Hosting or platform “preview protection” covering a domain you meant to make public.

SEO impact

Googlebot doesn’t log in. A URL that returns 401 can’t be indexed, and if it was indexed, it’s removed. Search Console reports it as “Blocked due to unauthorized request (401).” Google’s status code documentation also warns not to use 401 or 403 to limit crawl rate, because those codes don’t slow crawling. For that, return 429 or 503 for a short time.

Used deliberately, 401 is useful: password protection is the most reliable way to keep a staging copy out of Google. A robots.txt disallow alone doesn’t stop a linked URL from being indexed without its content, and a noindex copied from staging to production can wipe a live site from results.

How to fix or set up 401

Remove unwanted protection

Check .htaccess files in the site root and parent folders for AuthType lines, your Nginx config for auth_basic, and your host’s dashboard for password or preview protection. Make sure asset folders aren’t covered.

Protect staging properly

# Apache (.htaccess)
AuthType Basic
AuthName "Staging"
AuthUserFile /etc/apache2/.htpasswd
Require valid-user

# Nginx
location / {
    auth_basic "Staging";
    auth_basic_user_file /etc/nginx/.htpasswd;
}

# create the password file
htpasswd -c /etc/nginx/.htpasswd editor

For APIs, send a WWW-Authenticate: Bearer challenge with your 401 so clients know to refresh their token, and use 403 when the token is valid but lacks the required permission.

How to check

Run public URLs through the HTTP status checker after every launch or migration; anything returning 401 is invisible to search engines. The HTTP header checker shows the WWW-Authenticate challenge, which tells you which layer is asking for credentials.

HTTP/1.1 401 Unauthorized
WWW-Authenticate: Basic realm="Staging", charset="UTF-8"
Content-Type: text/html; charset=utf-8
Cache-Control: no-store

Frequently asked questions