Overview
CAPTCHA leads Congo's secondary barriers ahead of Antibot
| Value | |
|---|---|
| WAF | 92.5% |
| Antibot | 18.2% |
| CAPTCHA | 24.7% |
| JavaScript | 38.3% |
| Rate Limiting | 12.3% |
| TLS | 17.7% |
WAF reaches 92.5% of Congo's 920 sites. CAPTCHA (24.7%) is the leading secondary barrier. Antibot (18.2%), TLS (17.7%), JavaScript (38.3%), and Rate Limiting (12.3%) complete the picture.
Barrier Landscape
CAPTCHA leads Congo's secondary barriers despite JavaScript's reach
| Value | |
|---|---|
| WAF | 92.5% |
| Antibot | 18.2% |
| CAPTCHA | 24.7% |
| JavaScript | 38.3% |
| Rate Limiting | 12.3% |
| TLS | 17.7% |
| Has robots.txt | 79.6% |
| Blocks AI crawlers | 12.8% |
WAF covers 92.5% of sites. Despite JavaScript reaching 38.3%, CAPTCHA (24.7%) is the most-deployed secondary barrier by count. Antibot sits at 18.2% and TLS at 17.7%.
One in twenty Congo sites carry no barriers at all
| % of sites | |
|---|---|
| 0 barriers | 4.8% |
| 1 barrier | 33.8% |
| 2 barriers | 32% |
| 3 barriers | 17.4% |
| 4 barriers | 7.2% |
| 5 barriers | 4.3% |
| 6 barriers | 0.5% |
4.8% of Congo sites deploy zero barriers — the highest zero-barrier share in this set. The modal count is one barrier (33.8%), with 32.0% carrying two and 29.4% stacking three or more.
Most Congo sites are Simple to access
| Mean recommended tier | |
|---|---|
| All sites | 1.6 |
The browser median sits at Tier 1, indicating more than half of Congo's sites require only the simplest retrieval method. A mean of 1.59 confirms the majority of the sample stays within the lowest two tiers.
Nearly nine in ten Congo sites are Simple or Easy to access
| % of sites | |
|---|---|
| 1 - Simple | 56% |
| 2 - Easy | 32.9% |
| 3 - Moderate | 8.3% |
| 4 - Complex | 2.2% |
| 5 - Advanced | 0.7% |
56.0% of Congo sites resolve at Tier 1 (Simple) and 32.9% at Tier 2 (Easy). Tier 3 accounts for 8.3%. Tiers 4 and 5 together cover fewer than 3% of the 920-domain sample.
robots.txt
AI crawlers are the most-named bot category in Congo's robots.txt files
| % of sites | |
|---|---|
| AI crawlers | 15.3% |
| SEO / search indexing | 13.2% |
| SEO audit tools | 13% |
| Commercial scrapers | 6.8% |
| Social / content scrapers | 6.6% |
| Archive / research crawlers | 6.1% |
| Monitoring / uptime bots | 0.3% |
AI crawlers are named on 15.3% of all Congo sites — the most-mentioned category. SEO/search indexing (13.2%) and SEO audit tools (13.0%) are close behind. Monitoring and uptime bots appear on just 0.3% of sites.
Uniform per-agent rules are Congo's most common robots.txt posture
| % of sites | |
|---|---|
| Uniform per-agent rules (no path mixing) | 39.1% |
| Path-level mixed rules for same agent | 35.8% |
| Allow-only (no disallow rules) | 1.7% |
| Mixed AI posture (block some, allow others) | 2% |
| Blanket Disallow: / | 2.8% |
79.6% of Congo sites have a robots.txt. Of all sites, 39.1% apply uniform per-agent rules and 35.8% use path-level mixed rules. Allow-only postures are rare at 1.7%.
GPTBot faces the most Disallow rules in Congo's robots.txt
| Disallowed | Allowed (whitelisted) | |
|---|---|---|
| GPTBot | 10.2% | 1.8% |
| CCBot | 6.8% | 0.4% |
| ClaudeBot | 7.4% | 0.8% |
| Google-Extended | 7.8% | 1.4% |
| Bytespider | 6.5% | 0.3% |
| meta-externalagent | 2.8% | 0.2% |
| ChatGPT-User | 4.7% | 1.8% |
| PerplexityBot | 5.5% | 1.7% |
| Applebot-Extended | 3.7% | 0.2% |
| OAI-SearchBot | 3.2% | 1.2% |
| anthropic-ai | 3.9% | 0.8% |
GPTBot appears in Disallow rules on 10.2% of all Congo sites — the most blocked AI agent. Google-Extended follows at 7.8% and ClaudeBot at 7.4%. GPTBot and ChatGPT-User share the top Allow position at 1.8%.
Multiple Congo sectors all name at least one bot to restrict
| % of sites | |
|---|---|
| Design | 100% |
| Newspapers | 100% |
| Publishing | 100% |
| Adult | 90% |
| Automotive | 90% |
| Computer & Video Games | 90% |
| Consulting | 90% |
| Entertainment | 90% |
| General | 90% |
| Manufacturing | 90% |
| Mass Media | 90% |
| Online Services | 90% |
| Retail | 90% |
| Wholesalers | 90% |
| Writing & Editing | 88.9% |
Design, Newspapers, and Publishing each reach 100%: every site names at least one bot in a Disallow rule. Adult, Automotive, Computer & Video Games, Consulting, Entertainment, General, Manufacturing, Mass Media, Online Services, Retail, and Wholesalers follow at 90%.
Congo's Automotive sites sites lead in explicitly allowing crawlers
| % of sites | |
|---|---|
| Automotive | 90% |
| Airlines | 80% |
| Consulting | 80% |
| Travel & Tourism | 80% |
| Computer Hardware | 70% |
| Photography | 70% |
| Publishing | 70% |
| Renewable Energy | 70% |
| Retail | 70% |
| Textiles & Nonwovens | 62.5% |
| Architecture | 60% |
| Hospitality | 60% |
| Jewellery & Luxury | 60% |
| Music | 60% |
| Recruitment & Staffing | 60% |
Automotive leads explicit bot Allow rules at 90% — above Airlines, Consulting, and Travel & Tourism which each reach 80%. The top five sectors all sit at 70% or above, indicating widespread selective allowlisting.
Industry Patterns
Computer Hardware, Retail, and Hospitality are Congo's hardest sectors
| WAF | Antibot | CAPTCHA | JavaScript | Rate Limit | TLS | robots.txt | Blocks AI | Tier | |
|---|---|---|---|---|---|---|---|---|---|
| Computer Hardware | 90% | 10% | 10% | 10% | 20% | 50% | 90% | 0% | Tier 3 |
| Retail | 100% | 10% | 0% | 70% | 20% | 10% | 90% | 50% | Tier 3 |
| Hospitality | 100% | 20% | 0% | 50% | 20% | 40% | 70% | 10% | Tier 3 |
| Consumer Electronics | 70% | 40% | 20% | 30% | 30% | 40% | 80% | 10% | Tier 2 |
| Apparel & Fashion | 100% | 0% | 30% | 80% | 20% | 20% | 70% | 10% | Tier 2 |
| Human Resources | 100% | 60% | 50% | 40% | 50% | 30% | 80% | 10% | Tier 2 |
| Jewellery & Luxury | 80% | 30% | 20% | 40% | 40% | 60% | 80% | 20% | Tier 2 |
| Pharmaceuticals | 100% | 0% | 10% | 40% | 10% | 0% | 70% | 10% | Tier 2 |
| Maritime | 70% | 20% | 0% | 50% | 10% | 30% | 70% | 10% | Tier 2 |
| Transport & Logistics | 70% | 20% | 10% | 30% | 10% | 30% | 70% | 0% | Tier 2 |
| Real Estate | 100% | 20% | 10% | 50% | 20% | 10% | 80% | 20% | Tier 2 |
| Recruitment & Staffing | 100% | 40% | 50% | 20% | 0% | 20% | 90% | 20% | Tier 2 |
| Automotive | 100% | 0% | 10% | 50% | 20% | 20% | 90% | 30% | Tier 2 |
| Banking | 100% | 0% | 10% | 10% | 0% | 0% | 80% | 0% | Tier 2 |
| Finance | 80% | 20% | 10% | 20% | 10% | 10% | 70% | 30% | Tier 2 |
Computer Hardware is Congo's hardest sector with a mean tier of 2.95 and a median of 3 — more than half its sites require at least Tier 3. Retail (mean 2.60) and Hospitality (mean 2.40) also reach median Tier 3.
Congo's most accessible sectors share a floor mean tier of 1.70
| WAF | Antibot | CAPTCHA | JavaScript | Rate Limit | TLS | robots.txt | Blocks AI | Tier | |
|---|---|---|---|---|---|---|---|---|---|
| Mail & Delivery | 70% | 20% | 20% | 20% | 20% | 40% | 40% | 0% | Tier 2 |
| Music | 100% | 40% | 40% | 60% | 20% | 20% | 100% | 20% | Tier 2 |
| Publishing | 90% | 20% | 20% | 30% | 10% | 10% | 100% | 30% | Tier 2 |
| Science | 100% | 30% | 20% | 50% | 40% | 10% | 70% | 10% | Tier 2 |
| Sports | 100% | 0% | 30% | 50% | 10% | 10% | 90% | 40% | Tier 2 |
| Travel & Tourism | 90% | 20% | 0% | 60% | 20% | 20% | 80% | 30% | Tier 2 |
| Design | 100% | 30% | 40% | 50% | 30% | 40% | 100% | 30% | Tier 2 |
| Food & Beverages | 100% | 20% | 40% | 20% | 0% | 0% | 90% | 20% | Tier 2 |
| Airlines | 90% | 20% | 0% | 50% | 10% | 20% | 90% | 30% | Tier 2 |
| Computer & Video Games | 100% | 20% | 20% | 50% | 10% | 10% | 100% | 20% | Tier 2 |
| Ecology & Environment | 100% | 30% | 10% | 50% | 0% | 10% | 90% | 10% | Tier 2 |
| Import & Export | 100% | 20% | 20% | 30% | 0% | 30% | 90% | 10% | Tier 2 |
| Machinery | 90% | 10% | 20% | 50% | 30% | 10% | 80% | 10% | Tier 2 |
| Non-Profit | 90% | 20% | 10% | 40% | 10% | 10% | 70% | 0% | Tier 2 |
| Photography | 90% | 30% | 50% | 20% | 20% | 90% | 80% | 30% | Tier 2 |
Seven sectors cluster at the floor mean tier of 1.70; the remaining eight range between 1.80 and 1.90. Photography stands out within this group: 90% of its sites deploy TLS — the highest TLS rate of any sector in the Congo dataset.