Matthew Widdop
Matty is a Digital SEO Executive at Intelligency, helping our clients to improve their digital performance. Matty is currently studying for his Level 3 Multi-Channel Marketer apprenticeship and has completed a Sport Journalism Degree at the University of Huddersfield.
The number one priority for technical SEOs in 2026 should be making sure that AI bots can find and retrieve your content. AI is growingly rapidly and not only are AI chatbots such as Claude and ChatGPT continuing to grow in popularity but Google is also slowly becoming dominated by AI; first with AI Overviews and now with the introduction of AI Mode, making sure AI chatbots can retrieve your content is therefore, more paramount than ever.
Crawlability and Indexation
The main focus for SEOs when technically optimising for AI chatbots should be to make sure the crawlability and indexability of your site is fully optimised. This essentially means that AI can go through the content on your website and retrieve any relevant information that gives it an insight into what your website is about and therefore which queries it should use to place you in front of the correct audience.
The most imperative step you can take to make sure your content can be easily crawled and indexed is to make sure your robots.txt file allows for AI crawlers. A robots.txt file is A robots.txt file is a plain text file placed in a website’s root directory to tell web crawlers and search engine bots which pages or folders they can and cannot access. A robots.txt file can inadvertently block crawlers as old robots.txt files can have rules written in them that block content retrieval bots. Make sure you have parsed through your robots.txt and it is fully optimised to allow AI crawlers to access your content.
An XML sitemap also helps AI crawlers crawl your site efficiently. This is basically what it says in the name; a map of your site that tells crawlers how your site is structured and what content needs visiting and is important. You can make sure your sitemap is routinely up to date by submitting it to Google Search Console whenever you make changes.
JavaScript rendering is also a problem when it comes to crawlability for AI visibility. None of the main AI crawlers render JavaScript which means if your site is built using heavy JavaScript injections, AI chatbots will not be able to understand your content properly. To fix this make sure you are using HTML for critical content.
Schema Markup
Schema markup is structured data injected into the HTML of a webpage to give crawlers and search engines bots more information about what your web page represents. Schema is important for AI because it helps give specific information about each page on your website to AI chatbots.
It’s important when you’re applying a schema to your site that it is applied correctly. Common mistakes people make include using the wrong type of schema. Such as using product schema for a normal service page or using irrelevant schema just because it’s available. A simple way to explain correct vs incorrect application is: good schema accurately describes what the page and its entities actually represent; bad schema tries to tell search engines something different, exaggerated, irrelevant, or outdated.
Ultimately, optimising for AI doesn’t mean abandoning traditional technical SEO principles. In many cases, it means getting the fundamentals right. If AI crawlers can’t access your pages, retrieve your content or clearly understand the entities and information on them, even the best content strategy will struggle to achieve consistent AI visibility.
Start by making your website as easy as possible for both search engines and AI systems to access and interpret. Keep your robots.txt and XML sitemap clean, make important content available within the HTML rather than relying heavily on JavaScript, and use accurate schema markup to provide additional context.
