State of Web Access2026

Religion & Belief has unusually high robots.txt discipline — 89% of sites publish a file, well above the 76% cross-sector norm. Yet only 10% name an AI crawler to restrict, suggesting the policy habit predates the AI access debate.

State of Web Access 2026
Religion & Belief · Barrier Profile · 2026

JavaScript leads Religion & Belief's secondary barrier stack above the norm

JavaScript leads Religion & Belief's secondary barrier stack above the norm
Value
WAF96%
Antibot22%
CAPTCHA26%
JavaScript32%
Rate Limiting19%
TLS13%

WAF at 96% is the dominant barrier. JavaScript (32%) leads secondary coverage. The remaining barriers sit materially lower, giving Religion & Belief a narrow but defined secondary layer.

100 sites
State of Web Access 2026
Religion & Belief · Access Barrier Adoption · 2026

WAF leads Religion & Belief at 96% — JavaScript is the main secondary layer

WAF leads Religion & Belief at 96% — JavaScript is the main secondary layer
Value
WAF96%
Antibot22%
CAPTCHA26%
JavaScript32%
Rate Limiting19%
TLS13%
Has robots.txt89%
Blocks AI crawlers10%

WAF dominates at 96%. JavaScript (32%) leads active secondary barriers, followed by CAPTCHA (26%). The remaining three barriers all sit at or below 22% — a concentrated rather than broadly stacked profile.

100 sites
State of Web Access 2026
Religion & Belief · Barrier Count Distribution · 2026

Single-barrier configurations dominate Religion & Belief — mostly WAF alone

Single-barrier configurations dominate Religion & Belief — mostly WAF alone
% of sites
0 barriers2%
1 barrier41%
2 barriers29%
3 barriers14%
4 barriers4%
5 barriers9%
6 barriers1%

41% of Religion & Belief sites carry exactly one barrier — almost always WAF. Two-barrier stacks add 29%. Just 28% of sites layer three or more defences.

100 sites
State of Web Access 2026
Religion & Belief · Recommended Scraping Tier · 2026

Religion & Belief sites are predominantly Simple to access

Religion & Belief sites are predominantly Simple to access
Mean recommended tier
All sites1.32

A mean recommended tier of 1.32 and a median of Simple place Religion & Belief toward the easier end of the sectors surveyed. 77% of sites are Simple to access; 5% require Moderate or above.

100 sites
State of Web Access 2026
Religion & Belief · Tier Distribution · 2026

Simple tier covers more than three-quarters of Religion & Belief sites

Simple tier covers more than three-quarters of Religion & Belief sites
% of sites
1 - Simple77%
2 - Easy18%
3 - Moderate3%
4 - Complex0%
5 - Advanced2%

Simple (77%) is the most common tier. Simple and Easy together account for 95% of sites, while 3% land at Moderate and 2% at Complex or Advanced.

100 sites
State of Web Access 2026
Religion & Belief · robots.txt Coverage · 2026

Most Religion & Belief sites publish a robots.txt — coverage above the norm

Most Religion & Belief sites publish a robots.txt — coverage above the norm
% of sites
Publish a valid robots.txt89%

89% of Religion & Belief sites publish a valid robots.txt. The 11% without a file leave crawl policy undefined — a lower gap than the cross-sector average of around 24%.

100 sites
State of Web Access 2026
Religion & Belief · Named Agents in robots.txt · 2026

Religion & Belief sites name a handful of agents in robots.txt — few commit to comprehensive policies

Religion & Belief sites name a handful of agents in robots.txt — few commit to comprehensive policies
% of sites
029%
150%
2–410%
5–97%
10–152%
16+2%

The mean of 3.2 named agents reflects limited bot-policy investment in Religion & Belief. 29% of sites name no agents; 50% name exactly one. Only 2% maintain a list of 16 or more named bots.

100 sites
State of Web Access 2026
Religion & Belief · Bot Category Mentions in robots.txt · 2026

AI crawlers draw the most robots.txt attention in Religion & Belief

AI crawlers draw the most robots.txt attention in Religion & Belief
% of sites
AI crawlers11%
SEO / search indexing4%
SEO audit tools3%
Commercial scrapers1%
Social / content scrapers3%
Archive / research crawlers5%
Monitoring / uptime bots0%

AI crawlers are named in 11% of Religion & Belief robots.txt files, followed by Archive / research crawlers at 5%. All figures are percentages of all 100 sites including those with no robots.txt.

100 sites
State of Web Access 2026
Religion & Belief · robots.txt Rule Posture Types · 2026

Uniform per-agent rules dominate Religion & Belief robots.txt files

Uniform per-agent rules dominate Religion & Belief robots.txt files
% of sites
Uniform per-agent rules57%
Path-level mixed rules27%
Allow-only (no Disallow rules)4%
Mixed AI posture1%
Blanket Disallow: /1%

Uniform per-agent rules lead at 57%, with path-level mixed rules at 27%. Mixed AI posture (1%) and blanket Disallow (1%) are minimal.

100 sites
State of Web Access 2026
Religion & Belief · Bot Agent Disallow vs Allow · 2026

AI crawlers are among the most-targeted agents in Religion & Belief robots.txt

AI crawlers are among the most-targeted agents in Religion & Belief robots.txt
DisallowedAllowed (whitelisted)
*60%33%
GPTBot8%1%
Google-Extended4%1%
Bytespider4%
ClaudeBot4%
meta-externalagent4%
Amazonbot3%
Applebot-Extended3%
CCBot3%
Googlebot2%1%
WebCopier3%
wget3%
anthropic-ai2%1%
Baiduspider2%
HTTrack2%
MSIECrawler2%
Slurp2%
GetRight/4.22%
HTTrack 3.02%
Harvest/1.52%
LinkextractorPro2%
NetAnts2%
Offline Explorer2%
SiteSnagger2%
Web Image Collector2%

Values are percentages of all 100 Religion & Belief sites surveyed. The wildcard (*) leads at 60% Disallow. Among named agents, GPTBot, Google-Extended lead the Disallow list — reflecting direct targeting of AI training and indexing bots.

100 sites surveyed · 89 with robots.txt
State of Web Access 2026
Nonprofit · Access Barriers · 2026

Religion & Belief sits mid-table within the Nonprofit cohort

Religion & Belief sits mid-table within the Nonprofit cohort
WAFAntibotCAPTCHAJavaScriptRate LimitTLSrobots.txtBlocks AITier
Museums95%27%32%43%22%13%78%13%Tier 1
Nonprofit Organizations96%18%19%45%24%17%83%10%Tier 2
Religion & Belief96%22%26%32%19%13%89%10%Tier 1
Social & Charitable Organizations94%22%31%38%20%11%79%5%Tier 1

Across Nonprofit, Religion & Belief's barrier rates are broadly in line with peers. WAF at 96%, JavaScript at 32%, and AI blocking at 10% all sit near the cohort median. Tier difficulty (mean 1.23) is below the group average.

11,100 landing pages