State of Web Access2026

Somalia's robots.txt behaviour has a clear priority: AI crawlers are the most-cited bot category at 16.1% of all sites, ahead of SEO and audit tools. Active bot-control barriers tell a quieter story — only 3.3% of sites carry no barriers at all, but rate limiting reaches just 11.7%.

State of Web Access 2026
Somalia · Barrier Profile · 2026

JavaScript is Somalia's most common secondary barrier

JavaScript is Somalia's most common secondary barrier
Value
WAF94.3%
Antibot21.3%
CAPTCHA24%
JavaScript41.4%
Rate Limiting11.7%
TLS18.8%

WAF covers 94.3% of Somalia's 990 surveyed sites — the highest barrier rate in the country profile. JavaScript follows as the leading secondary barrier at 41.4%, with CAPTCHA at 24.0%.

990 sites
State of Web Access 2026
Somalia · Barrier Adoption · 2026

JavaScript tops Somalia's active secondary defences

JavaScript tops Somalia's active secondary defences
Value
WAF94.3%
Antibot21.3%
CAPTCHA24%
JavaScript41.4%
Rate Limiting11.7%
TLS18.8%
Has robots.txt79.8%
Blocks AI crawlers14.2%

WAF is present on 94.3% of Somalia sites. JavaScript, the strongest secondary barrier, reaches 41.4%. Rate Limiting at 11.7% is the least-deployed of the six barriers.

990 sites
State of Web Access 2026
Somalia · Barrier Stacking · 2026

Two barriers is Somalia's most common configuration

Two barriers is Somalia's most common configuration
% of sites
0 barriers3.3%
1 barrier30.5%
2 barriers35.2%
3 barriers18.1%
4 barriers8.7%
5 barriers3.6%
6 barriers0.6%

Only 3.3% of Somalia sites carry no access barriers — the lowest zero-barrier rate in this set. The largest group, at 35.2%, deploys exactly two barriers. Sites with three or more total 31.0%.

990 sites
State of Web Access 2026
Somalia · Recommended Zyte API Tier · 2026

Most Somalia sites are Simple to access

Most Somalia sites are Simple to access
Mean recommended tier
All sites1.6

The median recommended tier across Somalia's 990 sites is Tier 1. The mean of 1.62 reflects a share of sites pulling above Tier 1, including 9.1% at Tier 3 and 2.3% at Tiers 4 or 5.

990 sites
State of Web Access 2026
Somalia · Zyte API Tier Distribution · 2026

Nearly nine in ten Somalia sites are Simple or Easy to access

Nearly nine in ten Somalia sites are Simple or Easy to access
% of sites
1 - Simple52%
2 - Easy36.6%
3 - Moderate9.1%
4 - Complex1.5%
5 - Advanced0.8%

Tier 1 accounts for 52.0% of Somalia sites and Tier 2 for 36.6%. Tier 3 applies to 9.1%, while Tiers 4 and 5 together reach 2.3% of the total.

990 sites
State of Web Access 2026
Somalia · Bot Category Mentions in <code>robots.txt</code> · 2026 · 990 Sites

AI crawlers are Somalia's most-cited robots.txt bot category

AI crawlers are Somalia's most-cited robots.txt bot category
% of sites
AI crawlers16.1%
SEO / search indexing13.3%
SEO audit tools12.5%
Commercial scrapers8%
Social / content scrapers6.7%
Archive / research crawlers5.4%
Monitoring / uptime bots0.3%

AI crawlers appear in robots.txt on 16.1% of all Somalia sites — the most-mentioned category. SEO/search indexing follows at 13.3% and SEO audit tools at 12.5%.

990 sites
State of Web Access 2026
Somalia · <code>robots.txt</code> Rule Posture Types · 2026 · 990 Sites

One in five Somalia sites publishes no robots.txt

One in five Somalia sites publishes no robots.txt
% of sites
Uniform per-agent rules (no path mixing)40.2%
Path-level mixed rules for same agent34.2%
Allow-only (no disallow rules)2.4%
Mixed AI posture (block some, allow others)2.6%
Blanket Disallow: /2.7%

40.2% of Somalia sites use uniform per-agent rules; 34.2% mix path-level rules for the same agent. The remaining 20.2% publish no robots.txt at all.

990 sites
State of Web Access 2026
Somalia · AI Agent <code>Disallow</code> vs <code>Allow</code> · 2026 · 990 Sites Surveyed

GPTBot leads Somalia's Disallow and Allow rates

GPTBot leads Somalia's Disallow and Allow rates
DisallowedAllowed (whitelisted)
GPTBot12.3%2.3%
CCBot9.5%1%
ClaudeBot9.2%1.2%
Google-Extended9.7%1.8%
Bytespider8.1%0.6%
meta-externalagent3.6%0.1%
ChatGPT-User6.1%2.3%
PerplexityBot6.4%2.2%
Applebot-Extended5.8%0.5%
OAI-SearchBot4.2%2.1%
anthropic-ai5.7%1.2%

GPTBot is named in a Disallow directive on 12.3% of all Somalia sites — the highest block rate. It also ties ChatGPT-User for the most explicit Allow entries, both at 2.3%.

990 sites surveyed · 790 with robots.txt
State of Web Access 2026
Somalia · Named-Bot <code>Disallow</code> Rate by Industry · Top 15 · 2026

Multiple Somalia sectors all name at least one bot to restrict

Multiple Somalia sectors all name at least one bot to restrict
% of sites
Agriculture100%
Design100%
Entertainment100%
Retail100%
Telecom100%
Accounting & Auditing90%
Adult90%
Airlines90%
Computer & Video Games90%
Consulting90%
Consumer Electronics90%
Foreign Language90%
Furniture90%
General90%
Mass Media90%

Agriculture, Design, Entertainment, Retail, and Telecom each have 100% of their sites naming at least one bot in a Disallow rule — the highest rate across all Somalia sectors.

Most categories n=10 — each site = 10 percentage points · 990 Somalia sites
State of Web Access 2026
Somalia · Explicit Bot <code>Allow</code> / Whitelist Rate by Industry · Top 15 · 2026

Somalia's Airlines and Retail sites lead in explicitly allowing crawlers

Somalia's Airlines and Retail sites lead in explicitly allowing crawlers
% of sites
Airlines80%
Retail80%
Photography70%
Publishing70%
Science70%
Training & Cert.70%
Foreign Language60%
IT Services60%
Import & Export60%
Mental Health60%
Outsourcing60%
Reference60%
Renewable Energy60%
Software & Dev60%
Sports60%

Airlines and Retail are tied at 80.0% — the highest share of Somalia sites naming at least one bot in an explicit Allow rule. Photography, Publishing, Science, and Training follow at 70.0%.

Most categories n=10 — each site = 10 percentage points · 990 Somalia sites
State of Web Access 2026
Somalia · Scraping Difficulty by Industry · Top 15 by Mean Tier · 2026

Computer Hardware, Human Resources, and Real Estate are Somalia's hardest sectors

Computer Hardware, Human Resources, and Real Estate are Somalia's hardest sectors
WAFAntibotCAPTCHAJavaScriptRate LimitTLSrobots.txtBlocks AITier
Computer Hardware80%10%0%20%10%30%70%0%Tier 3
Human Resources100%70%50%30%50%30%70%10%Tier 3
Real Estate100%10%10%70%50%20%40%0%Tier 3
Apparel & Fashion90%0%20%70%20%20%90%10%Tier 2
Automotive100%30%10%50%50%20%60%10%Tier 2
Retail100%10%0%70%10%10%100%50%Tier 2
Hospitality90%30%10%60%20%40%40%0%Tier 2
Jewellery & Luxury100%30%10%40%50%30%80%10%Tier 2
Airlines90%20%10%60%10%20%90%40%Tier 2
Beauty & Cosmetics80%50%40%50%40%30%80%10%Tier 2
Consumer Electronics70%20%10%30%10%40%90%20%Tier 2
Mail & Delivery70%20%10%20%0%30%80%0%Tier 2
Recruitment & Staffing100%30%40%20%0%10%80%20%Tier 2
Transport & Logistics70%30%20%20%10%40%70%10%Tier 2
Adult100%40%30%80%20%20%90%10%Tier 2

Computer Hardware, Human Resources, Real Estate, and Retail all share a mean of 2.50 — the highest in Somalia's top 15. Computer Hardware, Human Resources, and Real Estate are the only three to reach a Moderate median.

Top 15 industries by mean Zyte browser tier · 990 sites
State of Web Access 2026
Somalia · Scraping Difficulty by Industry · Bottom 15 by Mean Tier · 2026

Eight Somalia sectors share the most accessible floor

Eight Somalia sectors share the most accessible floor
WAFAntibotCAPTCHAJavaScriptRate LimitTLSrobots.txtBlocks AITier
Sporting Goods90%40%50%10%20%10%70%10%Tier 2
Biotech100%40%60%70%30%30%80%30%Tier 2
Design100%20%30%40%20%30%100%30%Tier 2
Maritime80%20%0%40%10%30%80%10%Tier 2
Reference100%30%10%50%20%0%70%30%Tier 2
Restaurants100%30%10%40%10%0%50%0%Tier 2
Food & Beverages100%20%40%60%10%10%70%40%Tier 2
Ecology & Environment100%20%20%50%0%20%80%20%Tier 2
Education100%20%10%30%10%10%80%10%Tier 2
Import & Export100%20%30%40%0%30%80%10%Tier 2
Manufacturing100%20%30%60%0%40%80%10%Tier 2
Military90%10%10%20%0%20%60%0%Tier 2
Museums100%30%20%60%10%10%70%30%Tier 2
Science100%20%10%40%30%10%80%10%Tier 2
Telecom90%0%10%60%0%20%100%20%Tier 2

Ecology & Environment, Education, Import & Export, Manufacturing, Military, Museums, Science, and Telecom all record a mean tier of 1.80 — the lowest across Somalia's bottom 15 sectors.

Bottom 15 industries by mean Zyte browser tier · 990 sites