HTTP Headers Checker: Read What Your Server Really Sends

Every page and file arrives with a block of HTTP response headers before the first byte of content. Visitors never see them, but search engines read them closely: a single header can stop a page from being indexed, name its canonical URL, decide how long it is cached and whether it is compressed.

This HTTP header checker requests the URL and shows the raw response headers exactly as returned: the status line, server, cache-control, content-type, content-encoding, X-Robots-Tag, Link and Strict-Transport-Security. The SEO-relevant ones come with a short explanation, so a stray noindex or missing compression stands out right away.

How to check HTTP headers

1. Enter the URL

   Paste any page, image or PDF address. Servers often treat file types differently, so test the exact URL you care about.

2. Read the raw response

   You see the status code and every header name and value in the order the server sent them.

3. Act on the notes

   The SEO-relevant headers are explained, and values worth fixing, such as an unexpected X-Robots-Tag, are highlighted.

Reading a response, line by line

HTTP/2 200
content-type: text/html; charset=UTF-8
content-encoding: br
cache-control: public, max-age=300
link: <https://www.example.com/pricing>; rel="canonical"
x-robots-tag: noindex
strict-transport-security: max-age=31536000; includeSubDomains
vary: Accept-Encoding
server: nginx

• Status line: the 200 that crawlers act on, whatever the page text says.
• content-type: HTML in UTF-8. A wrong charset garbles accented characters in titles and snippets.
• content-encoding: br (Brotli) or gzip means the response was compressed. Google’s crawlers support gzip, deflate and Brotli.
• cache-control: how long browsers and CDNs may reuse the response.
• link and x-robots-tag: indexing instructions. Here the page would be dropped from the index despite its canonical.
• strict-transport-security: forces HTTPS on future visits (HSTS).

Header names are case-insensitive (RFC 9110); HTTP/2 and HTTP/3 send them in lowercase.

Headers that override your HTML

Two headers carry indexing instructions, and they also work for files that have no <head>, such as PDFs and images:

• X-Robots-Tag accepts the same rules as the robots meta tag (noindex, nofollow, nosnippet, max-snippet, unavailable_after) and can target one crawler, as in X-Robots-Tag: googlebot: noindex. See Google’s robots meta tag and X-Robots-Tag specification.
• Link can declare rel="canonical" and hreflang alternates for non-HTML files.

To keep PDFs out of search on Apache, and the same on Nginx:

<FilesMatch "\.pdf$">
  Header set X-Robots-Tag "noindex"
</FilesMatch>

location ~* \.pdf$ {
    add_header X-Robots-Tag "noindex";
}

The classic accident is the reverse: a staging rule like Header set X-Robots-Tag "noindex" carried over to production, silently deindexing every page. It never appears in the page source, only in the headers. The indexability checker tests it together with meta robots and robots.txt.

Caching and compression headers

HTML usually gets a short max-age, or no-cache with revalidation, so updates show quickly. Versioned static files, whose names change with their content (app.3f9a1c.css), can be cached for a year with cache-control: public, max-age=31536000, immutable.

Validators help crawling too. With an ETag or Last-Modified header, Google’s crawlers can send a conditional request and receive a short 304 Not Modified instead of the full page (Google crawler overview). Expect vary: Accept-Encoding on compressed responses, and vary: User-Agent if you serve different HTML to phones and desktops on the same URL. If HTML, CSS or JavaScript arrives without content-encoding, enable Brotli or gzip; the page speed test shows what that costs you.

Headers that say too much

server: Apache/2.4.41 (Ubuntu) or x-powered-by: PHP/7.4.3 tell attackers exactly which versions to look up. Hide them with ServerTokens Prod on Apache, server_tokens off; on Nginx and expose_php = Off in php.ini. For the protective headers you should add instead, such as Content-Security-Policy and Referrer-Policy, run the security headers checker.

Tips for checking headers

• Test one URL of each type (a page, an image, a PDF, a stylesheet), because servers often apply different rules to each.
• Compare the response from your CDN with the one from your origin; CDN rules can add, strip or rewrite headers.
• Look for duplicates, such as two cache-control or two X-Robots-Tag lines set by both a plugin and the server.
• From a terminal, curl -s -D - -o /dev/null https://example.com prints the headers of a real GET request; some servers answer HEAD requests (curl -I) differently.
• After changing server config, purge the CDN cache before re-testing, or you may be looking at an old copy.

Limits to keep in mind

We show the response to one request from our server, without cookies or a login. Headers can vary by user agent, country, device, cookie or CDN edge, and some firewalls answer automated requests differently from browsers. Only response headers are shown, not the request headers your browser sends; for those, open Chrome DevTools, choose the Network tab and click the document request.

Last updated Oct 30, 2026 by the Yes SEO team.