The AI Pioneers - October 2026

The Model Grounding Paradox: the builders of AI discovery engines run corporate websites that AI systems can barely read.

Do the companies building frontier AI make their own websites ready for AI search?

The five companies in this edition are spending heavily to change how people find information. Their models, assistants and search products are teaching buyers to ask a machine instead of browsing a website. We asked a simple question: how well do these companies prepare their own corporate websites for that shift?

The answer is a paradox. On the three domains we could measure, every site lets AI crawlers in, yet none publishes an llms.txt context file and none exposes a structured company identity that an answer engine can rely on. They are open, but not explained. An AI system can visit, but it has to guess what it is looking at.

The contrast inside Google is the clearest signal. Its research lab, DeepMind, scores 67 on exactly the same checks, more than three times the score of google.com. Readiness is being built inside product and research teams, not at the corporate level where investors, journalists, regulators and enterprise buyers start their research.

Two pioneers could not be scored. OpenAI's site redirected and challenged our scanner, and SpaceX publishes no crawler policy at all. We report these as findings, not as zeros: when a company's own site cannot be independently checked, AI systems face the same limit, and fall back on secondary sources they do not control.

Synthesis

None of the measured pioneers publish an llms.txt file on their main domain.

All three measured sites allow AI crawlers, but none expose structured company identity.

Google DeepMind scores 67 on the same checks, more than three times the parent domain.

Directives

Separate access from understanding: Allowing AI crawlers is only the first step. Leadership should ask whether AI systems can also tell who the company is, what it offers and which pages are authoritative. Today, none of the measured pioneers answer that question on their main domain.

Bring sub-brand practice up to corporate level: Where a research lab or product team has already done the work, as DeepMind has, the same standard should apply to the corporate domain. That is the page AI systems cite when buyers, journalists and regulators ask about the company.

Make protection deliberate, not accidental: Bot protection and missing crawler policies both reduce what AI systems can verify. Neither is wrong on its own, but each should be a documented decision that weighs security against being described accurately in AI answers.

Cohort

Anthropic (anthropic.com): 20. Anthropic's site is open and technically clean, earning full marks on foundations. It is a well-built conventional website, not one written for machines: there is no AI context file and no structured company identity.

OpenAI (openai.com) - not scored. Our scanner was redirected and then challenged at the edge, so no reliable public evidence could be collected this month. We do not estimate scores for sites we cannot read.

Google (google.com): 19. google.com works as a search tool, not a corporate brand page. It carries almost no machine-readable description of Google as a company, and leaves that job to sub-brands and external profiles.

Meta (meta.com): 20. Meta champions open AI models, but its corporate site follows a conventional pattern: crawlable and well built, yet not described for AI systems.

SpaceX (spacex.com) - not scored. The site publishes neither a robots.txt nor an llms.txt file, so there is no stated policy for AI crawlers at all. Access is left to each crawler's default behaviour.