Technical SEO: what it is, what to fix first, and what to skip

Technical SEO is the work that lets search engines and AI systems fetch, read, index and cite a site’s pages. It covers what sits under the words: status codes, redirects, robots rules, canonical tags, rendering, speed and structured data. Get it wrong and good content never gets the chance to rank.

On a typical B2B site of a few hundred pages, the order is short: whatever stops pages being indexed, then wrong status codes and duplicate addresses, then speed and markup. That is where technical SEO for search and AI visibility starts, and much of the usual checklist can wait.

What technical SEO covers, and where it stops

Technical SEO is one of three jobs that are easy to blur. The simplest test is where the fix happens: in the code and on the server, in the words on the page, or on other people’s sites.

Some work sits on the line. Schema markup is technical in how it gets added and on-page in what it says, so it needs a developer and a writer. Internal links are on-page, but a menu that crawlers cannot follow is a technical fault.

How a page gets into results and AI answers

Every page passes through the same steps. A crawler fetches it, a renderer runs its JavaScript, the index stores what it found, and the engine chooses it, or not, for a result. A fault at one step stops every step after it, which is why technical work comes first.

AI answers sit on the same steps. Google says a page must be indexed and eligible to show with a snippet to appear in AI Overviews or AI Mode, with no extra technical requirements. Bing says it and Copilot rest on the same crawling, indexing and ranking foundation.

AI companies also run crawlers of their own. Cloudflare’s 2025 Year in Review found AI bots other than Googlebot made 4.2% of the HTML requests it saw, with Googlebot alone at 4.5%. Some fetch for training, some for search, and some only when a person asks.

That last kind visits on someone’s behalf, and robots.txt may not stop it. OpenAI says its rules may not apply to ChatGPT-User, and Google says the same of user-triggered fetchers such as Google-Agent, the agent that browses and acts for its users.

What to fix first, in order

Rank technical faults by what they cost, not by how many a crawl report lists. A page that cannot be indexed loses everything; a slow page loses some of its visitors. On a site of a few hundred pages, this order covers almost every fault that matters.

Pages that cannot be indexed

Start with Search Console’s Page indexing report, which lists the pages Google knows about and why each one is or is not indexed. For any page that matters, URL Inspection shows what Google fetched and whether it is allowed to index it.

The usual culprit is a noindex tag left over from a staging site. The other is a robots.txt block, which is the wrong tool: Google says robots.txt exists to manage crawler traffic, not to keep a page out of its results. Noindex does that.

JavaScript adds a trap. Google’s guide, clarified in December 2025, warns that removing a noindex with a script may not work. If you want a page indexed, keep noindex out of the code the server sends.

Wrong status codes and broken redirects

A live page should answer 200, a moved page a permanent redirect, and a missing page 404. A missing page that answers 200 with a “not found” message is what Google calls a soft 404, and it shows up in the same Page indexing report.

The code matters for rendering too. Google’s JavaScript guide says that when a page answers with anything other than 200, rendering might be skipped, so content a script adds to an error page may never be read.

This site had the reverse problem. Its old WordPress build was hacked, and search engines still listed spam at addresses like /?⁠8995103. The static host ignored the query string and answered each one with the home page and a 200, so a short routing rule now makes them return 404.

The move itself needed 94 permanent redirects, one for each old address with a current page to go to, so links earned by the old site still count. Google’s site move guide, updated in August 2026, says permanent redirects lose no PageRank.

One address for every page

The same page often answers at several addresses: with and without www, over http and https, with and without a trailing slash, with tracking tags on the end. Pick one version, redirect the others to it, and point canonical tags, sitemaps and internal links at that one.

Google weighs the signals: a redirect and a canonical tag are strong, a sitemap entry weak. It asks JavaScript sites not to change the canonical a page was served with. For new addresses, its URL guide, updated in October 2026, recommends hyphens between words, not underscores.

Content that needs JavaScript to appear

Google renders JavaScript, and its changelog said in March 2026 that it has done so for multiple years now. Its guide still recommends rendering on the server, or before the page is served, because that is faster and not every bot can run JavaScript.

So send the words, links, canonical tag and structured data in the HTML itself. A marketing site rarely needs a script to show its copy. Every page on this one is plain HTML, so a technical fix is an edit to the page source, checked before it ships.

Size is the newer limit. Google’s March 2026 post on how Googlebot fetches pages says it reads up to 2MB of each page, headers included, and stops there. Google expects almost no page to reach it; bloated inline scripts, styles and images are how pages do.

Speed measured on real visitors

Core Web Vitals measure real visits: how fast the main content appears (LCP), how quickly the page responds to a click or tap (INP), and how much the layout jumps (CLS). Search Console rates pages good when 75% of visits manage 2.5 seconds, 200 milliseconds and 0.1.

Google says good scores, with the rest of page experience, align with what its ranking systems seek to reward. Just over half of sites have them: Chrome’s field data for September 2026 rated 55.9% of 18.4 million sites good on all three.

Loading is the usual weak point, with 68.3% good on LCP against 85.4% on INP. Read field data in Search Console or PageSpeed Insights; a Lighthouse score is a lab test on one machine, useful for finding a cause but not what Google reports.

Structured data that matches the page

Structured data tells engines what a page is: an article, an organization, a product, a trail of breadcrumbs. Mark up only what readers can see. Google’s policies, updated in July 2026, say misleading markup can cost a page its rich results, though not its ranking.

Check which results still exist before building for one. FAQ rich results stopped appearing in Google on 7 May 2026, and Google dropped practice problem markup in November 2025.

Sitemaps and fast discovery

A sitemap lists the pages you want found. Google uses its lastmod dates only when they are consistently accurate and ignores priority and changefreq, so a plugin that stamps today’s date on every page wastes the one field that counts.

Bing and a few other engines also take IndexNow, a ping sent the moment a page is added, changed or removed. Bing’s February 2026 post on AI Performance says it keeps content fresh across search and AI experiences. Google is not among them, so keep the sitemap.

What a smaller site can leave alone

Much of the usual checklist is written for sites with millions of pages. Google’s own documentation takes several items off the list for a site of a few hundred:

• Crawl budget. Google’s guide is for sites of over a million pages that change weekly, or over 10,000 that change daily. Below that, it says an up-to-date sitemap and a regular look at Page indexing are enough.
• Log file analysis, mostly. Server logs matter when crawl budget does. Google even says a site with fewer than a thousand pages should not need its Crawl stats report, the summary of Googlebot’s requests.
• Sitemap priority and changefreq. Google ignores both.
• Meta keywords, word counts and heading order. Google does not use the keywords tag, length alone does not affect ranking, and headings out of order do not matter to Search, though they help screen readers.
• A duplicate content penalty. One page at several addresses is inefficient, Google says, but will not cause a manual action. Fix it to concentrate signals, not out of fear.
• hreflang on a one-language site. It tells Google which language or regional version of a page to show. With one version, there is nothing to choose between.
• A perfect Lighthouse score. Field data is what counts, and past “good” there is little left to win.

None of it is wrong to do. It is just later than the work above.

Technical SEO for AI search: what changes

Less than the pitches suggest. AI Overviews and AI Mode draw on the same index, so everything above applies unchanged. The new work is choosing which AI crawlers may visit, and checking that they can.

Each AI company now splits its crawlers by job. OpenAI says a site that blocks OAI-SearchBot will not be shown in ChatGPT search answers, while blocking GPTBot only opts out of training, and that a robots.txt change takes about 24 hours to reach its search.

Anthropic says its bots, Claude-SearchBot included, honor robots.txt. Google-Extended governs Gemini’s use of your pages, and Google says it has no effect on inclusion or ranking in Search.

The common mistake is a rule for one bot that quietly changes what another sees, because a crawler obeys only the most specific group that names it. A firewall or a host’s bot setting can block the same crawlers while robots.txt says yes, so check both.

Two newer items are optional. Google says llms.txt files are not needed for Search and neither help nor harm there, though other systems do read them. And Google plans to honor a shared AI training preference in robots.txt, one signal for many AI companies, in early 2027.

A technical SEO checklist you can run for free

Every fault above shows up in free reports from Google and Bing. No paid crawler is needed to find them on a site of a few hundred pages, and these reports show what the engines themselves saw.

The last row is new. Since 31 August 2026 every site’s Search Console shows its impressions in AI Overviews and AI Mode, and Bing’s report, in preview since February 2026, shows how often Copilot and Bing’s AI summaries cite each page.

Most new faults arrive with a release, so run the indexing and status checks after every deploy as well as on the calendar. A fault caught the same day costs a fix; one found a month later costs the traffic in between.