State of Web Access2026

Gambia's robots.txt files are notably active on AI policy — AI crawlers are the single most-named bot category, appearing on 16.4% of all sites. Yet active technical defences are light: Rate Limiting reaches just 11.9% of sites, among the lowest secondary barriers in the profile.

State of Web Access 2026
Gambia · Barrier Profile · 2026

JavaScript leads Gambia's secondary barriers, ahead of CAPTCHA

JavaScript leads Gambia's secondary barriers, ahead of CAPTCHA
Value
WAF93.4%
Antibot20.1%
CAPTCHA24.5%
JavaScript36.8%
Rate Limiting11.9%
TLS18.3%

WAF reaches 93.4% of Gambian sites. JavaScript (36.8%) leads secondary barriers, followed by CAPTCHA at 24.5% and TLS at 18.3%. Antibot stands at 20.1% and Rate Limiting at 11.9%.

854 sites
State of Web Access 2026
Gambia · Barrier Adoption · 2026

JavaScript tops Gambia's active secondary defences

JavaScript tops Gambia's active secondary defences
Value
WAF93.4%
Antibot20.1%
CAPTCHA24.5%
JavaScript36.8%
Rate Limiting11.9%
TLS18.3%
Has robots.txt82.4%
Blocks AI crawlers14.5%

WAF is present on 93.4% of Gambian sites. JavaScript (36.8%) leads secondary barriers, ahead of CAPTCHA at 24.5% and TLS at 18.3%. Rate Limiting is the lowest secondary at 11.9%.

854 sites
State of Web Access 2026
Gambia · Barrier Stacking · 2026

One barrier is Gambia's most common site configuration

One barrier is Gambia's most common site configuration
% of sites
0 barriers3.5%
1 barrier34.9%
2 barriers32.3%
3 barriers16.3%
4 barriers8.5%
5 barriers4.2%
6 barriers0.2%

34.9% of Gambian sites carry exactly one barrier — the modal count. Zero-barrier sites are the lowest in this group at 3.5%. Sites with three or more barriers total 29.2%.

854 sites
State of Web Access 2026
Gambia · Recommended Zyte API Tier · 2026

Most Gambia sites are Simple to access

Most Gambia sites are Simple to access
Mean recommended tier
All sites1.6

A browser-based mean recommended tier of 1.57 and median of 1 confirm that more than half of Gambian sites are retrievable at the simplest level without browser rendering.

854 sites
State of Web Access 2026
Gambia · Zyte API Tier Distribution · 2026

Most Gambia sites are Simple to access

Most Gambia sites are Simple to access
% of sites
1 - Simple56.1%
2 - Easy33.6%
3 - Moderate7.7%
4 - Complex2%
5 - Advanced0.6%

Tier 1 covers 56.1% of Gambian sites — the highest Tier 1 share in this group. Tier 2 adds 33.6%. Tier 3 accounts for 7.7%; Tiers 4 and 5 together total 2.6%.

854 sites
State of Web Access 2026
Gambia · Bot Category Mentions in <code>robots.txt</code> · 2026 · 854 Sites

AI crawlers top Gambia's robots.txt bot categories

AI crawlers top Gambia's robots.txt bot categories
% of sites
AI crawlers16.4%
SEO / search indexing12.6%
SEO audit tools14.8%
Commercial scrapers7.8%
Social / content scrapers6.6%
Archive / research crawlers4.8%
Monitoring / uptime bots0.5%

AI crawlers appear in 16.4% of all Gambian sites. SEO audit tools follow closely at 14.8% — the smallest gap between the top two categories. SEO/search indexing stands at 12.6%.

854 sites
State of Web Access 2026
Gambia · <code>robots.txt</code> Rule Posture Types · 2026 · 854 Sites

Uniform rules are Gambia's most common robots.txt posture

Uniform rules are Gambia's most common robots.txt posture
% of sites
Uniform per-agent rules (no path mixing)40.9%
Path-level mixed rules for same agent34%
Allow-only (no disallow rules)4.2%
Mixed AI posture (block some, allow others)2.5%
Blanket Disallow: /3.2%

40.9% of all Gambian sites apply uniform per-agent rules. Path-level mixed rules cover 34.0%. Allow-only posture — Disallow-free files — accounts for 4.2% of all Gambian sites.

854 sites
State of Web Access 2026
Gambia · AI Agent <code>Disallow</code> vs <code>Allow</code> · 2026 · 854 Sites Surveyed

GPTBot faces the most Disallow rules in Gambia's robots.txt

GPTBot faces the most Disallow rules in Gambia's robots.txt
DisallowedAllowed (whitelisted)
GPTBot11.5%1.6%
CCBot8.7%0.6%
ClaudeBot8.4%0.8%
Google-Extended9%1.2%
Bytespider7.5%0.4%
meta-externalagent3%0.2%
ChatGPT-User6.6%2.3%
PerplexityBot6.6%2.5%
Applebot-Extended5.2%0.1%
OAI-SearchBot4.3%2.3%
anthropic-ai5%0.5%

GPTBot is named in a Disallow rule on 11.5% of all Gambian sites. Unusually, PerplexityBot tops the explicit Allow list at 2.5%, ahead of ChatGPT-User at 2.3%.

854 sites surveyed · 704 with robots.txt
State of Web Access 2026
Gambia · Named-Bot <code>Disallow</code> Rate by Industry · Top 15 · 2026

Multiple Gambia sectors all name at least one bot to restrict

Multiple Gambia sectors all name at least one bot to restrict
% of sites
Computer & Video Games100%
Consumer Electronics100%
Design100%
Entertainment100%
Graphic Design100%
Social & Charitable100%
Visual Art100%
Adult90%
Airlines90%
Furniture90%
General90%
Mass Media90%
Newspapers90%
Online Services90%
Photography90%

Computer & Video Games, Consumer Electronics, Design, Entertainment, Graphic Design, Social & Charitable, and Visual Art each reach 100% — every site names at least one bot in a Disallow rule.

Most categories n=10 — each site = 10 percentage points · 854 Gambia sites
State of Web Access 2026
Gambia · Explicit Bot <code>Allow</code> / Whitelist Rate by Industry · Top 15 · 2026

Airlines, Hospitality, and Social & Charitable share Gambia's top Allow rate

Airlines, Hospitality, and Social & Charitable share Gambia's top Allow rate
% of sites
Airlines80%
Hospitality80%
Social & Charitable80%
Market Research71.4%
Investment70%
Photography70%
Publishing70%
Retail70%
Training & Cert.70%
Architecture60%
Chemicals60%
Design60%
Ecology & Environment60%
Graphic Design60%
Medical Devices60%

Airlines, Hospitality, and Social & Charitable share Gambia's highest explicit Allow rate at 80%. Market Research follows at 71.4%; Investment, Photography, Publishing, Retail, and Training & Cert. sit at 70%.

Most categories n=10 — each site = 10 percentage points · 854 Gambia sites
State of Web Access 2026
Gambia · Scraping Difficulty by Industry · Top 15 by Mean Tier · 2026

Retail is the only Gambia sector at a Moderate median tier

Retail is the only Gambia sector at a Moderate median tier
WAFAntibotCAPTCHAJavaScriptRate LimitTLSrobots.txtBlocks AITier
Retail100%10%0%70%20%10%90%40%Tier 3
Computer Hardware90%20%20%30%10%20%80%10%Tier 2
Hospitality90%20%30%70%20%30%80%10%Tier 2
Apparel & Fashion100%0%30%60%20%30%80%10%Tier 2
Consumer Electronics80%50%40%50%20%40%100%20%Tier 2
Human Resources100%50%30%30%30%10%90%0%Tier 2
Real Estate90%10%20%40%20%20%80%20%Tier 2
Automotive80%20%10%20%30%30%70%30%Tier 2
Sporting Goods100%40%40%50%50%30%80%10%Tier 2
Banking90%30%10%10%10%20%70%0%Tier 2
Construction80%0%0%40%20%0%50%10%Tier 2
Design100%30%30%30%30%30%100%40%Tier 2
Reference87.5%37.5%25%37.5%12.5%12.5%62.5%37.5%Tier 2
Science100%30%10%50%30%10%70%10%Tier 2
Telecom90%20%20%70%0%30%90%20%Tier 2

Retail is the hardest sector in Gambia, with a mean recommended tier of 2.60 and a median of 3 — the only sector to reach that threshold. Computer Hardware and Hospitality follow at 2.35 and 2.30.

Top 15 industries by mean Zyte browser tier · 854 sites
State of Web Access 2026
Gambia · Scraping Difficulty by Industry · Bottom 15 by Mean Tier · 2026

Eight Gambia sectors share the most accessible floor

Eight Gambia sectors share the most accessible floor
WAFAntibotCAPTCHAJavaScriptRate LimitTLSrobots.txtBlocks AITier
Mail & Delivery62.5%25%12.5%12.5%0%25%62.5%0%Tier 2
Advertising & Mktg90%30%50%40%0%20%80%10%Tier 2
Airlines80%10%10%40%20%20%100%40%Tier 2
Ecology & Environment100%20%0%50%0%10%90%10%Tier 2
Finance80%30%0%10%10%10%70%20%Tier 2
Import & Export100%30%20%40%30%30%70%20%Tier 2
Music100%30%20%50%20%10%100%10%Tier 2
Adult100%30%30%60%10%20%90%10%Tier 2
Computer & Video Games100%20%10%50%10%10%100%10%Tier 2
Distance Learning100%40%60%40%30%10%70%30%Tier 2
Investment100%40%20%20%20%20%90%10%Tier 2
Non-Profit90%30%10%40%10%10%70%0%Tier 2
Photography90%40%40%10%20%80%90%30%Tier 2
Publishing90%30%30%50%20%0%90%60%Tier 2
Recruitment & Staffing80%10%40%30%10%20%70%0%Tier 2

Adult, Computer & Video Games, Distance Learning, Investment, Non-Profit, Photography, Publishing, and Recruitment & Staffing all record a mean recommended tier of 1.70 — Gambia's widest floor cluster.

Bottom 15 industries by mean Zyte browser tier · 854 sites