Methodology

We scanned 816 websites across UK, US and international markets, and scored each one using the PARSE Score - a 100-point assessment of how visible a website is to AI agents. Each site in this report was scanned once between March and April 2026. The 816 sites represent a cross-section of businesses across 11 verticals.

How verdicts work

Every scored site receives a verdict based on its PARSE Score.

Agentic Authority (85 to 100). The site meets the standard AI agents expect. Access is open, permissions are declared, and content is clearly structured. AI agents can find the site and understand what it offers.

Agentic Strain (60 to 84). The site is reachable but has identifiable gaps. AI agents can access it but encounter friction that reduces reliability or comprehension.

Digitally Anonymous (0 to 59). AI agents cannot reliably interact with the site. Access is blocked, content is too poorly structured to interpret, or both. A site in this band is effectively absent from the AI layer of the web.

A Critical fault overrides the calculated score regardless of everything else and forces a Digitally Anonymous verdict.

Sites with a high accumulation of Major faults receive a downgrade, shifting their verdict one band lower. No single fault here is critical, but together they create an experience that falls below the standard for AI agent interaction.

PARSE (Preparedness Assessment: Receiving-Side Evaluation) is AgentPratham's proprietary methodology for measuring how well a website is configured for AI agent access, comprehension and navigation.

The score runs from 0 to 100. It is deterministic: the same site, scanned under the same conditions, produces the same score. Every fault that fires reduces the score by a fixed penalty. No AI interpretation affects the score itself.

The score is accompanied by a verdict, one of three bands, that provides a qualitative anchor for the numerical result.

Each site in this dataset was scanned once using the AgentPratham scanner - a browser-based scanning engine that visits each URL as an AI agent would. The scanner reads publicly accessible pages, evaluates robots.txt and llms.txt declarations, checks structured data, and assesses content accessibility.

Scans were conducted between March and April 2026. Sites were not pre-screened for expected score. The scanner does not require authentication and does not attempt to access gated content.

Where a site was inaccessible or returned persistent errors during the scan window, it is excluded from the dataset.

The PARSE Score is evaluated across three pillars. Pillar scores are independently normalised display figures - they do not sum to the overall PARSE Score, which is calculated separately as 100 minus the total penalties fired across all criteria.

Technical Foundation

Covers the structural signals that make a site machine-readable: metadata completeness, structured data presence, canonical tags, performance signals, and content quality markers. This pillar holds the largest number of criteria and carries 40% of the scoring ceiling.

Actionable Handshake

Covers access and permission signals: robots.txt configuration, llms.txt adoption, authentication handling, and navigation clarity for automated agents. This pillar holds all Critical fault types. A Critical fault overrides the calculated score regardless of everything else and forces a Digitally Anonymous verdict. Carries 35% of the ceiling.

Semantic Truth

Covers the accuracy and consistency of site signals: declared crawl policies versus live behaviour, trust and authority indicators, and entity signals. Semantic Truth has a smaller maximum penalty exposure than the other pillars, which means individual faults here have a proportionally larger effect on the pillar sub-score. Carries 25% of the ceiling.

This dataset is a snapshot. Each site was scanned once. A site's score reflects its configuration at the time of scanning - changes made after the scan date are not captured.

The scanner assesses publicly accessible content only. It does not evaluate gated, authenticated, or dynamically generated content requiring a user session. Sites that present different content to crawlers versus users will score against what the crawler sees.

This dataset covers 816 domains across 11 industry verticals. It is not a random sample and makes no claim to statistical representativeness of the business landscape as a whole. Verticals were selected based on domain availability and crawlability at the time of sampling.

Score comparisons across verticals are directional only. Structural differences between sectors affect fault prevalence in ways unrelated to deliberate AI readiness investment.

If you believe your site's score is incorrect due to a technical error in the scanning process, contact us at contact@agentprime.co.uk with the subject line "Score correction request." Include your domain and a description of the specific criterion you believe was incorrectly assessed.

We do not accept score change requests on the basis of site improvements made after the scan date. The benchmark is a point-in-time record. Sites rescanned in a future wave will receive an updated score.

We do not share raw scan data or individual fault breakdowns for sites assessed as part of the benchmark. Clients who purchase a PARSE Score report for their own domain receive their full breakdown.

The sites assessed in this benchmark are publicly accessible websites. The scanner accessed only publicly facing pages - equivalent to a user or automated tool browsing the site without authentication.

The data published in this benchmark relates to the technical configuration of websites, not to individuals. No personal data was collected or processed in the course of scanning.

AgentPrime Ltd is registered with the UK Information Commissioner's Office. ICO registration number: ZC114543.

AgentPratham does not retain raw page content from scanned sites. The scanner extracts structured signals from page content during the scan and discards the source material. Only the derived assessment outputs are retained.

If you believe that data relating to your organisation has been included in error, or wish to query how your site was assessed, contact us at contact@agentprime.co.uk.

For information about how AgentPratham handles personal data collected via this website, see our Privacy policy at agentprime.co.uk/privacy.