Part 5 in our BrightonSEO 2026 series. You can read the hub post here, Part 2 on why your traffic is falling here, Part 3 on getting your brand mentioned by AI here, and Part 4 on audience-first content strategy here.
Continuing our series of posts following on from BrightonSEO, the world’s biggest SEO conference, this week we are taking a look at the more technical aspect of SEO and AI Optimisation – another heavily covered theme throughout the conference. I attended some outstanding sessions around Technical SEO, however, this is where Paul’s strengths really come to the fore, so it was great to catch up at the end of each day and in between sessions to find out what he had discovered. Pulling our notes together in the post-conference debrief once I got back to New Zealand also helped me get a much better understanding of the current landscape and realise that a lot of the things we have been doing well for the past decade are still really relevant today – but that we must also embrace the new opportunities that exist whilst acknowledging the threats and dealing with them.
This is probably the most practically prescriptive post in this series so far, because technical SEO is an area where specific, concrete actions tend to have predictable, measurable outcomes. If there’s one overarching message to take from this post – and from the technical sessions at BrightonSEO more broadly – it’s this: the foundational rules haven’t changed. But there are new ways to break them that didn’t exist five years ago, and a new set of crawlers to consider alongside Google.
Let’s get into it.
The Most Important Thing We Learned: AI Crawlers Are Not Google
This sounds obvious when you say it out loud, but the implications run deeper than most businesses realise.
Google has spent the better part of a decade investing in its ability to render JavaScript, interpret dynamic content, and understand complex page architectures. It has largely, though not perfectly, solved the problem of JavaScript-heavy websites. ChatGPT, Claude, Gemini, Perplexity, and the crawlers that feed them have not.
Multiple speakers across both days of BrightonSEO made the same point: if your website relies on JavaScript to render its content, meaning the text, headings, products, or information only appear after JavaScript has executed in a browser, AI crawlers often cannot read it. To them, your page is essentially a blank screen. Your content, from their perspective, doesn’t exist.
Will Kennard’s (Vercel) session on auditing AI-generated (“vibe-coded”) websites was particularly illuminating here. As more businesses build websites using AI tools that generate JavaScript-heavy front-ends by default, the risk of creating sites that are invisible to AI search is growing rapidly. Judith Lewis and Daniel Liddle both flagged the same issue from different angles. The consensus across all three sessions: this is the single highest-priority technical issue for any business that cares about AI discoverability.
Think about what this means in practice. If you have a JavaScript-rendered product catalogue, a client-side-rendered blog, or a dynamic pricing table that only loads once a script fires, any of those could be completely invisible to the AI platforms that are increasingly influencing purchase decisions. You could have the best content in your category, and it would simply not be there, as far as an AI is concerned.
The fix isn’t always straightforward, but the diagnostic is: render your key pages with JavaScript disabled in your browser. If the content disappears, AI crawlers are likely having the same experience. That’s your starting point.
Paul’s take:“This came up in multiple sessions and roundtables, not just as a technical problem but as a strategic vulnerability. Businesses that moved to JavaScript-heavy frameworks in the last few years for perfectly good UX reasons now have a hidden technical debt in terms of AI visibility. The audit is the first step, but the remediation conversation, whether to server-side render, pre-render, or restructure the architecture, is one that needs to happen between the development team and the SEO team together, not in isolation.”
Core Web Vitals: Still Non-Negotiable
There was a point in the industry a couple of years ago when Core Web Vitals started to feel like old news – something that had been talked to death and was now just a hygiene item. BrightonSEO reminded us that this is the wrong framing.
Janaina Barreto-Romero’s (Oncrawl) session touched on Core Web Vitals in the context of AI crawlability. The finding: AI crawlers deprioritise slow and technically inaccessible pages, much as Google’s crawl budget prioritisation works. A page that loads slowly, has significant layout shift, or is unresponsive to interaction is a page that AI crawlers spend less time on, and potentially don’t index fully.
For a brand trying to establish AI visibility, this creates a hierarchy: fix the JavaScript rendering issues first, then ensure the pages you want to be cited are technically healthy by Core Web Vitals standards. These are the table stakes. Everything else – structured data, entity architecture, content optimisation – sits on top of this foundation.
The specific metrics to focus on if you haven’t audited recently:
- Largest Contentful Paint (LCP): how quickly does the main content of the page become visible? Under 2.5 seconds is the target.
- Interaction to Next Paint (INP): how responsive is the page to user interactions? This replaced First Input Delay in 2024 and is now the interaction metric Google measures.
- Cumulative Layout Shift (CLS): does the page jump around visually as it loads? A score under 0.1 is good.
These are measurable in Google Search Console, PageSpeed Insights, and a range of third-party audit tools. If you’re not checking them regularly, you’re flying blind on one of the fundamentals.
Structured Data and Schema Markup: The Signal AI Actually Rewards
This is the technical area where the BrightonSEO evidence was most encouraging, because it confirms that an investment we’ve been recommending to clients for years is paying off in ways that extend beyond traditional search.
Thomas Peham’s GEO testing session at Ryte was unusually rigorous for a conference talk: he ran controlled experiments across multiple variables and shared the results, including the ones that didn’t work as expected. On schema markup, his finding was clear: structured data produces significant and measurable improvements in Google visibility. Its direct impact on other LLMs is less certain right now, but the indirect effect – better Google performance, which leads to more citations, which influences AI training data – makes it worth implementing regardless.
Sam Davis’s session on entity architecture and LLM brand management went further. His argument, backed by data from his work at Yext, is that entity-rich structured data is the single most important technical signal for AI search performance when you look at the full picture. Not just on-site schema, but the entire ecosystem of structured entity data – the consistency of your business name, address, phone number, description, and categories across every platform that AI learns from.
The concept that crystallised this for Paul was the entity confidence score: a way of thinking about how confident an AI model is in its understanding of who your brand is and what it does, based on the consistency and breadth of signals across the web. An entity with Wikipedia coverage, a fully built-out LinkedIn company page, a consistent Google Business Profile, schema markup on the website, active Trustpilot and relevant directory profiles – all using exactly the same brand name, description, and category – has a high entity confidence score. An entity that appears inconsistently, under slightly different names, with different descriptions across different platforms, has a low one.
Paul’s take:“Darko Brzica’s session on links, mentions, and AI signals landed on this same point from the link-building angle. He showed that branded anchor text is now outperforming exact-match keywords in terms of link effectiveness – a 2.2% branded anchor share is consistently beating traditional exact-match approaches. What this tells us is that the web is moving towards recognising brand entities rather than keyword signals, and our technical and link-building strategies need to reflect that shift.”
The practical action here is Schema SameAs mapping – a schema property that explicitly connects your brand entity across platforms by linking to your profiles on Wikipedia, LinkedIn, Google Business Profile, Wikidata, social accounts, and relevant directories. It’s not complex to implement, but it’s something that the majority of small and mid-market websites don’t have in place. It’s now on our standard implementation checklist for every client.
The LLMs.txt Myth: Save Your Time
I want to spend a moment on something that has generated a lot of noise in SEO circles over the past 12 months, because Thomas Peham’s testing data at BrightonSEO cut through that noise very clearly.
LLMs.txt is a proposed web standard, similar in concept to robots.txt, that would allow website owners to provide instructions to AI crawlers about what content can and can’t be used for training or retrieval. It sounds logical. It generated a lot of excitement when it was first proposed. And yep, that included us! As SEOs, we are always searching for the marginal gains, and there was a time when it appeared that LLMs.txt files might be one of those gains, so it was good to put that to bed with some comprehensive, research-backed data.
Thomas’s data: 0.1% of AI bot traffic accesses LLMs.txt files. There is no positive correlation between having one and improved AI visibility.
Zero-point-one per cent. Not a rounding error. Not a methodology issue. The files simply aren’t being read meaningfully by AI crawlers at any scale that produces a measurable effect.
This doesn’t mean the concept is permanently dead – standards evolve, and if adoption increases significantly, the calculus might change. But right now, in mid-2026, implementing an LLMs.txt file as your primary AI optimisation strategy is a waste of development time. The sessions at BrightonSEO pointed consistently to structured data, entity consistency, third-party citations, and content quality as the levers that actually move the needle. Those are where the time and budget should go.
Log File Analysis: The Underused Intelligence Source
This was the technical topic I came away from BrightonSEO most energised about, partly because it’s something we haven’t been doing routinely for clients, and partly because the intelligence it surfaces is genuinely difficult to get any other way.
Your web server’s log files record every request made to your website, including requests from crawlers. This means they contain a complete, unfiltered record of which AI crawlers are visiting your site, how frequently, which pages they prioritise, and which pages they ignore. This is extraordinarily useful information if you’re trying to understand AI visibility, and it’s just sitting there in your server logs, largely unread.
Janaina Barreto-Romero has done detailed work in this area, and her findings were striking: different AI crawlers behave very differently. GPTBot (OpenAI’s crawler), Anthropic’s Claude crawler, Google’s AIO crawler, and Perplexity’s crawler all have distinct crawl patterns – different pages they prioritise, different depths they explore, different frequency patterns. Understanding which crawlers are visiting which pages tells you a great deal about where your content is and isn’t being indexed for AI retrieval.
Even more useful: tools like Oncrawl can now process log file data and provide estimated AI citation likelihood by page. You can see not just which pages are being crawled, but which ones are most likely being used as citation sources. That’s the kind of granular intelligence that lets you make targeted decisions about where to invest technical and content effort.
The diagnostic questions log file analysis can answer:
- Are AI crawlers finding my most important pages, or are they getting stuck on lower-priority content?
- Are there technical barriers (slow response times, redirect chains, blocked resources) that are reducing crawl efficiency for AI bots?
- Which pages are being crawled most frequently and does that match my content strategy priorities?
- Have I seen a change in AI crawl behaviour following a technical change or content update?
Getting started with log file analysis doesn’t require enterprise tooling. If you have access to your server logs (most hosting providers can provide them on request), you can do a basic analysis in a spreadsheet or using a free log file analyser. Jonathan Roberts’ session on automating SEO audits with Node.js, and Nicolas Montanares’ 80-checks automation framework, both referenced approaches to making log file analysis more efficient and repeatable and we’ve noted both sessions to work through in detail as we build out our own auditing processes.
Paul’s take:“Gus Pelogia’s session on MCP servers for SEO was adjacent to this but pointed in the same direction – the tools for automated, intelligent SEO analysis are maturing rapidly. His demonstration of using MCP workflows for keyword grouping, SERP scraping, and People Also Ask analysis alongside SEMrush showed what’s possible when you connect the right data sources. The manual effort that used to go into this kind of analysis is increasingly automatable, which frees up time for the interpretation and decision-making that actually requires human judgement.”
Entity Architecture: The Strategic Framework Tying It All Together
We’ve touched on entity architecture in earlier posts in this series, but it deserves more detailed treatment here because it’s fundamentally a technical discipline, not just a content or PR one.
Your brand entity is the sum of all the signals that tell AI models, and Google, who you are. It’s built from:
- Your official website and its structured data
- Your Google Business Profile
- Your LinkedIn company page
- Your Wikipedia and Wikidata entries (if they exist)
- Your profiles on review platforms (Trustpilot, Google Reviews, sector-specific review sites)
- Your profiles on professional directories relevant to your industry
- Your social media profiles
- Third-party mentions and citations that reference you by name
The critical technical principle is consistency. If your business name appears as “Digital Hothouse” on your website, “Digital Hothouse Ltd” on your Google Business Profile, “The Digital Hothouse” on LinkedIn, and “DH Agency” on a directory listing, you have four partial entity signals that don’t reinforce each other. The AI model’s confidence in who you are is divided and weakened.
Schema SameAs mapping, as mentioned earlier, is the technical mechanism for connecting these signals explicitly. But before you can implement it, you need to conduct an entity audit, systematically checking every major platform where your brand appears and ensuring the information is accurate, current, and consistently formatted.
Darko Brzica’s session provided a framework for entity stacking that Paul brought back from Brighton: starting with your own website (schema markup, structured data, consistent brand naming in metadata), then building outwards to Google Business Profile, Wikipedia/Wikidata, LinkedIn, key professional directories, and finally third-party validation platforms. Each layer strengthens the overall entity signal. The order matters – start with the platforms you fully control, then work outwards.
A specific action from Paul’s notes: if your brand doesn’t have a Wikipedia or Wikidata entry, and you meet the notability criteria, this is worth pursuing. Darko’s data showed that Wikipedia is one of the most frequently cited sources in AI responses for brands that have entries. It’s not always achievable for every business, but for established brands with media coverage and demonstrable impact, it’s worth investigating.
The Technical Audit Checklist We’re Now Running for Every Client
Pulling together the most consistent recommendations from the BrightonSEO technical sessions, here’s the priority order we’re working through for our clients:
Immediate priorities:
- JavaScript rendering audit – render key pages with JavaScript disabled. If content disappears, flag for remediation with the development team.
- AI crawler access check – review robots.txt to confirm AI crawlers (GPTBot, ClaudeBot, PerplexityBot, Google-Extended) are not blocked, unless there’s a specific reason to block them.
- Entity consistency audit – check brand name, description, and category data across Google Business Profile, LinkedIn, key directories, and social profiles. Identify and correct any inconsistencies.
- Schema SameAs implementation – connect your website’s structured data to your verified profiles across major platforms.
Priority maintenance (monthly):
- Core Web Vitals monitoring – Google Search Console provides ongoing Core Web Vitals data. Flag and address any pages with “poor” or “needs improvement” ratings.
- Structured data validation – run schema markup through Google’s Rich Results Test. Ensure no errors or warnings have crept in following content updates.
- Crawl health monitoring – check crawl errors in Google Search Console and ensure key pages are being indexed correctly.
Investigative (as capacity allows):
- Log file analysis – request log files from your hosting provider and run them through Oncrawl or a similar tool to understand AI crawler behaviour on your specific site.
- Page size audit – Jonathan Roberts flagged a new Google crawling threshold for pages over 2MB. Audit for any bloated pages and reduce where possible.
- XML sitemap audit – ensure your sitemap is accurate, up to date, and includes all the pages you want crawled, without including pages that should be excluded (thin content, parameter URLs, etc.).
This isn’t a complete technical SEO checklist – it’s a prioritised list of the items that BrightonSEO’s technical sessions most consistently identified as high-impact for AI visibility specifically. Many of these are things we’ve been doing as standard practice for years. The difference now is that we’re running them with AI crawlers in mind, not just Google.
The Reassuring Conclusion
I said this in the intro and I want to close by repeating it, because it genuinely came as a relief after a couple of days immersed in rapid-fire sessions about change and disruption: the fundamentals haven’t changed.
A website that is technically clean, loads quickly, uses structured data properly, and has consistent brand signals across the web will perform well for Google. It will also perform well for AI crawlers. It will also be more likely to be cited by AI platforms in their answers. The audience has expanded. The fundamentals remain.
What has changed is the consequence of getting the basics wrong. In a traditional search world, a JavaScript rendering issue or inconsistent entity data was a missed opportunity. In an AI-driven world, where AI platforms are increasingly mediating the first impression your brand makes on a potential customer, those same issues can mean your brand simply doesn’t exist in the conversation.
That’s the non-negotiable part. And it’s entirely within your control to fix.
If you’d like us to run a technical audit for your website, looking specifically at the AI visibility factors covered in this post, get in touch with us here. It’s one of the most practical conversations we’re having with clients right now.
Coming Up Next in This Series
In the next post, we turn to brand authority and digital PR, specifically why being mentioned and cited across the web has never mattered more, and how to build a systematic approach to earning the kind of third-party credibility that both search engines and AI platforms reward.
Gavin is Digital Hothouse’s SEO lead. He attended BrightonSEO 2026 with Director Paul Thornton.
