Overview
JavaScript is South Africa's most common secondary barrier
| Value | |
|---|---|
| WAF | 93.2% |
| Antibot | 20.8% |
| CAPTCHA | 23.1% |
| JavaScript | 41% |
| Rate Limiting | 12.6% |
| TLS | 18.5% |
WAF reaches 93.2% of South Africa's 1,111 sites. Among secondary barriers, JavaScript leads at 41.0%, followed by CAPTCHA at 23.1%, Antibot at 20.8%, TLS at 18.5%, and Rate Limiting at 12.6%.
Barrier Landscape
JavaScript tops South Africa's active secondary defences
| Value | |
|---|---|
| WAF | 93.2% |
| Antibot | 20.8% |
| CAPTCHA | 23.1% |
| JavaScript | 41% |
| Rate Limiting | 12.6% |
| TLS | 18.5% |
| Has robots.txt | 80.7% |
| Blocks AI crawlers | 14.4% |
WAF accounts for 93.2% of sites. JavaScript is the top secondary barrier at 41.0%, ahead of CAPTCHA (23.1%) and Antibot (20.8%). TLS reaches 18.5%. Rate Limiting is lowest among secondary barriers at 12.6%.
Two-barrier sites are the largest single group in South Africa
| % of sites | |
|---|---|
| 0 barriers | 4.7% |
| 1 barrier | 29.3% |
| 2 barriers | 35% |
| 3 barriers | 18.5% |
| 4 barriers | 8.3% |
| 5 barriers | 4% |
| 6 barriers | 0.3% |
35.0% of South Africa sites carry exactly two barriers — the single largest group. One barrier accounts for 29.3%. Three or more barriers total 31.1%. Sites with no barriers at all number just 4.7%.
Most South Africa sites are Easy to access
| Mean recommended tier | |
|---|---|
| All sites | 1.7 |
The mean browser-retrieval tier across 1,111 South Africa sites is 1.67, with a median of 2. A median above 1 indicates fewer than half of sites sit at Tier 1 (Simple).
Nearly half of South Africa sites are Simple to access
| % of sites | |
|---|---|
| 1 - Simple | 48.2% |
| 2 - Easy | 39.5% |
| 3 - Moderate | 10% |
| 4 - Complex | 1.7% |
| 5 - Advanced | 0.5% |
48.2% of South Africa sites sit at Tier 1 (Simple) and 39.5% at Tier 2 (Easy) — 87.7% combined. Tier 3 (Moderate) accounts for exactly 10.0%. Tiers 4 and 5 together hold just 2.2%.
robots.txt
AI crawlers lead South Africa's bot categories
| % of sites | |
|---|---|
| AI crawlers | 15.8% |
| SEO / search indexing | 11.2% |
| SEO audit tools | 13.4% |
| Commercial scrapers | 8% |
| Social / content scrapers | 5.7% |
| Archive / research crawlers | 4% |
| Monitoring / uptime bots | 0.5% |
AI crawlers appear in robots.txt rules on 15.8% of all 1,111 South Africa sites — more than any other category. SEO audit tools follow at 13.4% and SEO/search indexing at 11.2%.
Uniform rules are South Africa's most common robots.txt posture
| % of sites | |
|---|---|
| Uniform per-agent rules (no path mixing) | 42.3% |
| Path-level mixed rules for same agent | 32.9% |
| Allow-only (no disallow rules) | 2.4% |
| Mixed AI posture (block some, allow others) | 3.1% |
| Blanket Disallow: / | 2.9% |
42.3% of South Africa sites use uniform per-agent rules with no path mixing. Path-level mixed rules apply to 32.9%. Sites with no robots.txt account for 19.3% of all 1,111 sites.
GPTBot and ChatGPT-User share South Africa's highest Allow rate
| Disallowed | Allowed (whitelisted) | |
|---|---|---|
| GPTBot | 12% | 2.8% |
| CCBot | 8.8% | 1.1% |
| ClaudeBot | 8.5% | 1.2% |
| Google-Extended | 8.5% | 1.7% |
| Bytespider | 6.8% | 0.4% |
| meta-externalagent | 2.9% | 0.3% |
| ChatGPT-User | 6.7% | 2.8% |
| PerplexityBot | 6.2% | 2% |
| Applebot-Extended | 4.6% | 0.4% |
| OAI-SearchBot | 4.4% | 2.1% |
| anthropic-ai | 5.8% | 1.2% |
GPTBot carries a Disallow rule on 12.0% of all South Africa sites — the highest block rate. Both GPTBot and ChatGPT-User share the top allow rate at 2.8% each.
Multiple South Africa sectors all name at least one bot to restrict
| % of sites | |
|---|---|
| Civil Engineering | 100% |
| Computer & Video Games | 100% |
| Design | 100% |
| Retail | 100% |
| Visual Art | 100% |
| Adult | 90% |
| Comics & Animation | 90% |
| Consumer Electronics | 90% |
| Entertainment | 90% |
| Food & Beverages | 90% |
| General | 90% |
| Healthcare | 90% |
| Machinery | 90% |
| Mass Media | 90% |
| Medical Devices | 90% |
Civil Engineering, Computer & Video Games, Design, Retail, and Visual Art each have every sampled site naming at least one bot in a Disallow rule. Ten further sectors reach 90%.
South Africa's leading sectors explicitly allow crawlers
| % of sites | |
|---|---|
| Airlines | 70% |
| Architecture | 70% |
| Photography | 70% |
| Publishing | 70% |
| Science | 70% |
| Agriculture | 60% |
| Automotive | 60% |
| Civil Engineering | 60% |
| Graphic Design | 60% |
| Printing | 60% |
| Public Relations | 60% |
| Retail | 60% |
| Software & Dev | 60% |
| Sporting Goods | 60% |
| Sports | 60% |
Airlines, Architecture, Photography, Publishing, and Science each have 70% of sector sites naming at least one bot in an explicit Allow rule — the highest rate in South Africa's top-15 Allow list.
Industry Patterns
Jewellery & Luxury is South Africa's hardest sector to access programmatically
| WAF | Antibot | CAPTCHA | JavaScript | Rate Limit | TLS | robots.txt | Blocks AI | Tier | |
|---|---|---|---|---|---|---|---|---|---|
| Jewellery & Luxury | 100% | 10% | 10% | 30% | 50% | 50% | 60% | 0% | Tier 3 |
| Hospitality | 90% | 20% | 0% | 60% | 30% | 50% | 50% | 10% | Tier 3 |
| Equipment & Supplies | 90% | 40% | 30% | 50% | 20% | 10% | 50% | 0% | Tier 3 |
| Human Resources | 100% | 70% | 60% | 40% | 50% | 30% | 80% | 10% | Tier 2 |
| Retail | 90% | 20% | 0% | 60% | 10% | 20% | 100% | 40% | Tier 2 |
| Apparel & Fashion | 90% | 0% | 30% | 60% | 10% | 30% | 90% | 10% | Tier 2 |
| Beauty & Cosmetics | 90% | 20% | 10% | 60% | 50% | 0% | 80% | 10% | Tier 2 |
| Computer Hardware | 90% | 20% | 0% | 30% | 20% | 20% | 70% | 10% | Tier 2 |
| Consumer Electronics | 80% | 10% | 10% | 40% | 10% | 40% | 90% | 20% | Tier 2 |
| Customer Services | 100% | 10% | 30% | 40% | 20% | 20% | 80% | 10% | Tier 2 |
| Mail & Delivery | 70% | 40% | 10% | 20% | 0% | 20% | 80% | 0% | Tier 2 |
| Recruitment & Staffing | 100% | 40% | 70% | 10% | 0% | 20% | 90% | 30% | Tier 2 |
| Transport & Logistics | 70% | 40% | 10% | 20% | 0% | 20% | 80% | 0% | Tier 2 |
| Pharmaceuticals | 100% | 50% | 40% | 40% | 30% | 30% | 70% | 0% | Tier 2 |
| Wine & Spirits | 100% | 60% | 20% | 30% | 60% | 20% | 60% | 10% | Tier 2 |
Jewellery & Luxury tops South Africa's difficulty ranking with a mean tier of 2.90 and a median of 3. Hospitality follows at mean 2.70 (median 3). Equipment & Supplies reaches mean 2.40 at median 3.
Fifteen South Africa sectors share the most accessible floor at mean Simple.90
| WAF | Antibot | CAPTCHA | JavaScript | Rate Limit | TLS | robots.txt | Blocks AI | Tier | |
|---|---|---|---|---|---|---|---|---|---|
| Food & Beverages | 100% | 50% | 40% | 70% | 40% | 0% | 90% | 50% | Tier 2 |
| Design | 100% | 20% | 30% | 40% | 20% | 30% | 100% | 30% | Tier 2 |
| Public Safety | 100% | 30% | 40% | 60% | 20% | 40% | 90% | 10% | Tier 2 |
| Reference | 80% | 60% | 30% | 40% | 20% | 10% | 50% | 30% | Tier 2 |
| Sporting Goods | 90% | 10% | 30% | 20% | 20% | 30% | 90% | 10% | Tier 2 |
| Veterinary | 100% | 30% | 20% | 50% | 30% | 20% | 90% | 10% | Tier 2 |
| Airlines | 90% | 10% | 10% | 60% | 20% | 20% | 80% | 30% | Tier 2 |
| Banking | 100% | 40% | 10% | 20% | 10% | 20% | 80% | 0% | Tier 2 |
| Crafts | 100% | 30% | 30% | 50% | 20% | 10% | 80% | 20% | Tier 2 |
| Manufacturing | 80% | 10% | 10% | 30% | 0% | 40% | 70% | 0% | Tier 2 |
| Maritime | 90% | 10% | 0% | 50% | 10% | 30% | 90% | 10% | Tier 2 |
| Medical Devices | 100% | 0% | 10% | 30% | 10% | 0% | 100% | 10% | Tier 2 |
| Music | 100% | 40% | 40% | 60% | 20% | 20% | 90% | 20% | Tier 2 |
| Restaurants | 90% | 20% | 30% | 40% | 20% | 30% | 80% | 10% | Tier 2 |
| Textiles & Nonwovens | 90% | 0% | 30% | 30% | 0% | 10% | 90% | 10% | Tier 2 |
Nine sectors cluster at a mean tier of 1.90 — including Airlines, Banking, Manufacturing, and Restaurants. All bottom-15 sectors sit at median tier 2.