State of Web Access2026

Mail & Package Delivery has robots.txt coverage on 70% of sites, but almost none of it is targeted: only 3% name an AI crawler. The sector relies heavily on wildcard rules rather than deliberate named-bot policy.

State of Web Access 2026
Mail & Package Delivery · Barrier Profile · 2026

JavaScript leads Mail & Package Delivery's secondary barrier stack above the norm

JavaScript leads Mail & Package Delivery's secondary barrier stack above the norm
Value
WAF91%
Antibot15%
CAPTCHA19%
JavaScript37%
Rate Limiting17%
TLS15%

WAF at 91% is the dominant barrier. JavaScript (37%) leads secondary coverage. The remaining barriers sit materially lower, giving Mail & Package Delivery a narrow but defined secondary layer.

100 sites
State of Web Access 2026
Mail & Package Delivery · Access Barrier Adoption · 2026

WAF leads Mail & Package Delivery at 91% — JavaScript is the main secondary layer

WAF leads Mail & Package Delivery at 91% — JavaScript is the main secondary layer
Value
WAF91%
Antibot15%
CAPTCHA19%
JavaScript37%
Rate Limiting17%
TLS15%
Has robots.txt70%
Blocks AI crawlers3%

WAF dominates at 91%. JavaScript (37%) leads active secondary barriers, followed by CAPTCHA (19%). The remaining three barriers all sit at or below 17% — a concentrated rather than broadly stacked profile.

100 sites
State of Web Access 2026
Mail & Package Delivery · Barrier Count Distribution · 2026

Mail & Package Delivery's barrier counts are spread — no single configuration dominates

Mail & Package Delivery's barrier counts are spread — no single configuration dominates
% of sites
0 barriers6%
1 barrier35%
2 barriers35%
3 barriers13%
4 barriers5%
5 barriers6%
6 barriers0%

Barrier counts vary widely across Mail & Package Delivery sites. Single-barrier configurations are the modal count at 35%, but the distribution is broad. 24% of sites carry three or more.

100 sites
State of Web Access 2026
Mail & Package Delivery · Recommended Scraping Tier · 2026

Mail & Package Delivery sites are predominantly Simple to access

Mail & Package Delivery sites are predominantly Simple to access
Mean recommended tier
All sites1.30

A mean recommended tier of 1.30 and a median of Simple place Mail & Package Delivery toward the easier end of the sectors surveyed. 76% of sites are Simple to access; 4% require Moderate or above.

100 sites
State of Web Access 2026
Mail & Package Delivery · Tier Distribution · 2026

Simple tier covers more than three-quarters of Mail & Package Delivery sites

Simple tier covers more than three-quarters of Mail & Package Delivery sites
% of sites
1 - Simple75.8%
2 - Easy20.2%
3 - Moderate3%
4 - Complex0%
5 - Advanced1%

Simple (76%) is the most common tier. Simple and Easy together account for 96% of sites, while 3% land at Moderate and 1% at Complex or Advanced.

100 sites
State of Web Access 2026
Mail & Package Delivery · robots.txt Coverage · 2026

Around seven in ten of Mail & Package Delivery sites publish a robots.txt

Around seven in ten of Mail & Package Delivery sites publish a robots.txt
% of sites
Publish a valid robots.txt70%

70% of Mail & Package Delivery sites have a valid robots.txt. The remaining 30% leave crawl policy entirely undefined — a meaningful gap for a sector of this scale.

100 sites
State of Web Access 2026
Mail & Package Delivery · Named Agents in robots.txt · 2026

Most Mail & Package Delivery sites name one bot or none in their robots.txt

Most Mail & Package Delivery sites name one bot or none in their robots.txt
% of sites
049%
141%
2–46%
5–93%
10–150%
16+1%

49% of Mail & Package Delivery sites name zero agents and 41% name one. A mean of 1.1 is pulled up by a small number of sites with extensive policies, while the median stays at 1.

100 sites
State of Web Access 2026
Mail & Package Delivery · Bot Category Mentions in robots.txt · 2026

All bot categories receive minimal attention in Mail & Package Delivery robots.txt files

All bot categories receive minimal attention in Mail & Package Delivery robots.txt files
% of sites
AI crawlers3%
SEO / search indexing4%
SEO audit tools2%
Commercial scrapers0%
Social / content scrapers0%
Archive / research crawlers1%
Monitoring / uptime bots0%

No bot category exceeds 4% mention rate across Mail & Package Delivery sites. SEO / search indexing leads at 4%, reflecting very low overall engagement with named bot policy in this sector.

100 sites
State of Web Access 2026
Mail & Package Delivery · robots.txt Rule Posture Types · 2026

Uniform per-agent rules dominate Mail & Package Delivery robots.txt files

Uniform per-agent rules dominate Mail & Package Delivery robots.txt files
% of sites
Uniform per-agent rules44%
Path-level mixed rules20%
Allow-only (no Disallow rules)6%
Mixed AI posture0%
Blanket Disallow: /0%

Uniform per-agent rules lead at 44%, with path-level mixed rules at 20%. Mixed AI posture (0%) and blanket Disallow (0%) are minimal.

100 sites
State of Web Access 2026
Mail & Package Delivery · Bot Agent Disallow vs Allow · 2026

CCBot, GPTBot and Google-Extended lead named Disallow rules in Mail & Package Delivery robots.txt

CCBot, GPTBot and Google-Extended lead named Disallow rules in Mail & Package Delivery robots.txt
DisallowedAllowed (whitelisted)
*44%25%
Googlebot1%2%
CCBot3%
GPTBot3%
Google-Extended3%
EmailCollector2%
SemrushBot2%
SemrushBot-SA2%
Amazonbot2%
Applebot-Extended2%
Bytespider2%
ClaudeBot2%
meta-externalagent2%
AdsBot-Google0%2%
TermlyBot0%2%
ia_archiver1%
Ask Jeeves/Teoma1%
Yahoo Slurp1%
bingbot1%
YandexMobileBot1%
AhrefsBot1%
BLEXBot1%
Barkrowler1%
DataForSeoBot1%
Ezoicbot1%

Values are percentages of all 100 Mail & Package Delivery sites surveyed. The wildcard (*) leads at 44% Disallow. Among named agents, CCBot, GPTBot and Google-Extended share the top position, each named in a Disallow rule on 3% of all sites.

100 sites surveyed · 70 with robots.txt
State of Web Access 2026
Manufacturing & Logistics · Access Barriers · 2026

Mail & Package Delivery sits mid-table within the Manufacturing & Logistics cohort

Mail & Package Delivery sits mid-table within the Manufacturing & Logistics cohort
WAFAntibotCAPTCHAJavaScriptRate LimitTLSrobots.txtBlocks AITier
Aerospace & Defense87%17%26%36%21%15%67%6%Tier 1
Chemicals83%8%19%31%7%8%71%8%Tier 1
Civil Engineering89%13%22%38%19%14%60%5%Tier 1
Import & Export93%19%17%35%17%13%69%6%Tier 1
Machinery91%13%17%49%20%15%80%10%Tier 2
Mail & Package Delivery91%15%19%37%17%15%70%3%Tier 1
Manufacturing92%20%19%45%24%18%75%9%Tier 2
Maritime87%22%24%41%19%12%62%8%Tier 1
Metals & Mining87%26%30%39%25%16%72%4%Tier 1
Plastics & Polymers98%10%29%34%13%10%80%4%Tier 1
Transportation & Logistics78%15%16%35%17%18%67%2%Tier 1
Warehousing98%18%27%38%22%8%75%4%Tier 1

Across Manufacturing & Logistics, Mail & Package Delivery's barrier rates are broadly in line with peers. WAF at 91%, JavaScript at 37%, and AI blocking at 3% all sit near the cohort median. Tier difficulty (mean 1.32) is below the group average.

11,100 landing pages