Overview
JavaScript is Bangladesh's most common secondary barrier
| Value | |
|---|---|
| WAF | 93.2% |
| Antibot | 21.5% |
| CAPTCHA | 22.3% |
| JavaScript | 42.3% |
| Rate Limiting | 13.8% |
| TLS | 18.6% |
WAF covers 93.2% of 1,110 surveyed sites. JavaScript leads secondary barriers at 42.3%. CAPTCHA (22.3%) and Antibot (21.5%) are separated by just 0.8 percentage points — the slimmest gap among the six barriers.
Barrier Landscape
JavaScript tops Bangladesh's active secondary defences
| Value | |
|---|---|
| WAF | 93.2% |
| Antibot | 21.5% |
| CAPTCHA | 22.3% |
| JavaScript | 42.3% |
| Rate Limiting | 13.8% |
| TLS | 18.6% |
| Has robots.txt | 79.9% |
| Blocks AI crawlers | 13.2% |
WAF leads at 93.2% across 1,110 surveyed sites. JavaScript (42.3%) is the dominant secondary barrier, running 20 points ahead of CAPTCHA (22.3%). Antibot (21.5%) follows. Rate Limiting (13.8%) is the highest of the three lowest-deployed barriers.
Two-barrier sites are Bangladesh's most common configuration
| % of sites | |
|---|---|
| 0 barriers | 4.5% |
| 1 barrier | 28.3% |
| 2 barriers | 36% |
| 3 barriers | 17.7% |
| 4 barriers | 9.1% |
| 5 barriers | 4.1% |
| 6 barriers | 0.3% |
4.5% of sites present no detectable barriers. The modal configuration is exactly two barriers at 36.0%. Sites stacking three or more barriers account for 31.2% — meaning nearly a third of surveyed sites carry a multi-layer stack.
Most Bangladesh sites are Simple to access
| Mean recommended tier | |
|---|---|
| All sites | 1.7 |
Mean recommended tier is 1.70, median is 2. Tier 1 (Simple) covers 45.7% of sites — fewer than half. Tier 2 (Easy) adds 41.1%, making Easy the most consequential tier for reliable access.
More than eight in ten sites are Simple or Easy to access
| % of sites | |
|---|---|
| 1 - Simple | 45.7% |
| 2 - Easy | 41.1% |
| 3 - Moderate | 11% |
| 4 - Complex | 1.5% |
| 5 - Advanced | 0.7% |
Tier 1 (Simple) covers 45.7% of sites; Tier 2 (Easy) adds 41.1% — the highest Tier 2 share in this batch — for a combined 86.8%. Tier 3 (Moderate) accounts for 11.0%, Tier 4 for 1.5%, Tier 5 for 0.7%.
robots.txt
AI crawlers top Bangladesh's robots.txt bot categories
| % of sites | |
|---|---|
| AI crawlers | 15.1% |
| SEO / search indexing | 11.5% |
| SEO audit tools | 13.3% |
| Commercial scrapers | 7.4% |
| Social / content scrapers | 5.9% |
| Archive / research crawlers | 4.5% |
| Monitoring / uptime bots | 0.5% |
AI crawlers appear in robots.txt at 15.1% of all 1,110 surveyed sites. SEO audit tools rank second at 13.3%. SEO / search indexing (11.5%) sits 1.8 points further back — a larger gap than between first and second.
One in five Bangladesh sites publishes no robots.txt
| % of sites | |
|---|---|
| Uniform per-agent rules (no path mixing) | 40.9% |
| Path-level mixed rules for same agent | 33.1% |
| Allow-only (no disallow rules) | 3% |
| Mixed AI posture (block some, allow others) | 2.3% |
| Blanket Disallow: / | 2.9% |
20.1% of sites have no robots.txt. Uniform per-agent rules is the most common posture at 40.9% of all sites. Path-level mixed rules apply to 33.1%. Blanket Disallow: / appears at 2.9%.
GPTBot faces the most Disallow rules in Bangladesh's robots.txt
| Disallowed | Allowed (whitelisted) | |
|---|---|---|
| GPTBot | 10.9% | 2.3% |
| CCBot | 8.2% | 1% |
| ClaudeBot | 8% | 1.2% |
| Google-Extended | 7.9% | 1.5% |
| Bytespider | 7% | 0.4% |
| meta-externalagent | 3.1% | — |
| ChatGPT-User | 5.7% | 2.5% |
| PerplexityBot | 5.8% | 2.3% |
| Applebot-Extended | 4.8% | 0.3% |
| OAI-SearchBot | 4.3% | 2% |
| anthropic-ai | 4.9% | 0.8% |
GPTBot is named in a Disallow rule by 10.9% of all surveyed sites — the highest of any AI agent. ChatGPT-User has the highest explicit Allow rate at 2.5%, followed by GPTBot and PerplexityBot at 2.3%.
Multiple Bangladesh sectors all name at least one bot to restrict
| % of sites | |
|---|---|
| Automotive | 100% |
| Consumer Electronics | 100% |
| Design | 100% |
| Retail | 100% |
| Visual Art | 100% |
| Adult | 90% |
| Civil Engineering | 90% |
| Comics & Animation | 90% |
| Computer & Video Games | 90% |
| Entertainment | 90% |
| General | 90% |
| Insurance | 90% |
| Machinery | 90% |
| Mass Media | 90% |
| Newspapers | 90% |
Every site in Automotive, Consumer Electronics, Design, Retail, and Visual Art names at least one bot to restrict in robots.txt. Ten additional sectors sit at 90%, covering a wide range of industries at near-universal named-bot restriction.
Bangladesh's Airlines sites lead in explicitly allowing crawlers
| % of sites | |
|---|---|
| Airlines | 80% |
| Architecture | 70% |
| Automotive | 70% |
| Photography | 70% |
| Publishing | 70% |
| Retail | 70% |
| Science | 70% |
| Telecom | 70% |
| Veterinary | 70% |
| Events | 60% |
| Farms & Ranches | 60% |
| Graphic Design | 60% |
| Insurance | 60% |
| Investment | 60% |
| Non-Profit | 60% |
Airlines has the highest explicit bot Allow rate at 80% — the sole top-ranked sector. Architecture, Automotive, Photography, Publishing, Retail, Science, Telecom, and Veterinary follow at 70%, forming a broad eight-sector cluster.
Industry Patterns
Medical Devices is the hardest sector at near-Moderate mean difficulty
| WAF | Antibot | CAPTCHA | JavaScript | Rate Limit | TLS | robots.txt | Blocks AI | Tier | |
|---|---|---|---|---|---|---|---|---|---|
| Medical Devices | 100% | 20% | 10% | 60% | 20% | 0% | 90% | 0% | Tier 3 |
| Hospitality | 90% | 30% | 0% | 60% | 20% | 40% | 60% | 10% | Tier 3 |
| Real Estate | 100% | 30% | 30% | 70% | 60% | 20% | 50% | 10% | Tier 3 |
| Furniture | 100% | 50% | 10% | 20% | 10% | 50% | 70% | 10% | Tier 3 |
| Equipment & Supplies | 90% | 30% | 20% | 50% | 20% | 10% | 70% | 20% | Tier 3 |
| Jewellery & Luxury | 100% | 20% | 0% | 60% | 50% | 40% | 80% | 10% | Tier 2 |
| Retail | 100% | 10% | 0% | 70% | 10% | 10% | 100% | 50% | Tier 2 |
| Beauty & Cosmetics | 80% | 50% | 30% | 20% | 50% | 30% | 80% | 20% | Tier 2 |
| Computer Hardware | 90% | 20% | 0% | 30% | 20% | 20% | 70% | 10% | Tier 2 |
| Human Resources | 100% | 70% | 60% | 40% | 40% | 20% | 70% | 10% | Tier 2 |
| Apparel & Fashion | 100% | 10% | 20% | 60% | 20% | 20% | 70% | 10% | Tier 2 |
| Insurance | 80% | 40% | 20% | 50% | 10% | 30% | 90% | 0% | Tier 2 |
| Food & Beverages | 100% | 40% | 30% | 70% | 40% | 0% | 70% | 30% | Tier 2 |
| Consumer Electronics | 80% | 10% | 10% | 40% | 10% | 40% | 100% | 20% | Tier 2 |
| Crafts | 100% | 30% | 30% | 50% | 20% | 10% | 80% | 20% | Tier 2 |
Medical Devices averages a mean recommended tier of 2.70 with a median of 3 — the highest values of any sector. Hospitality and Real Estate both average 2.60, also with a median of 3. Industries ranked by mean recommended browser tier.
Four sectors share the most accessible floor
| WAF | Antibot | CAPTCHA | JavaScript | Rate Limit | TLS | robots.txt | Blocks AI | Tier | |
|---|---|---|---|---|---|---|---|---|---|
| Customer Services | 100% | 20% | 40% | 50% | 30% | 10% | 60% | 0% | Tier 2 |
| Restaurants | 90% | 40% | 40% | 60% | 40% | 20% | 60% | 20% | Tier 2 |
| Sporting Goods | 80% | 10% | 20% | 30% | 30% | 10% | 50% | 0% | Tier 2 |
| Wine & Spirits | 100% | 50% | 10% | 50% | 20% | 10% | 80% | 20% | Tier 2 |
| Design | 100% | 20% | 30% | 40% | 20% | 30% | 100% | 30% | Tier 2 |
| Mail & Delivery | 70% | 40% | 10% | 10% | 0% | 20% | 80% | 0% | Tier 2 |
| Maritime | 90% | 10% | 10% | 30% | 10% | 30% | 90% | 10% | Tier 2 |
| Public Safety | 100% | 40% | 40% | 60% | 20% | 40% | 90% | 10% | Tier 2 |
| Recruitment & Staffing | 100% | 50% | 60% | 20% | 10% | 30% | 70% | 20% | Tier 2 |
| Transport & Logistics | 70% | 40% | 10% | 10% | 0% | 20% | 80% | 0% | Tier 2 |
| Wholesalers | 80% | 20% | 20% | 40% | 30% | 30% | 70% | 10% | Tier 2 |
| Automotive | 100% | 10% | 0% | 20% | 20% | 20% | 100% | 20% | Tier 2 |
| Events | 100% | 10% | 10% | 80% | 20% | 0% | 80% | 10% | Tier 2 |
| Manufacturing | 80% | 10% | 10% | 30% | 0% | 40% | 70% | 0% | Tier 2 |
| Reference | 90% | 60% | 30% | 50% | 30% | 10% | 60% | 30% | Tier 2 |
Automotive, Events, Manufacturing, and Reference each average a mean recommended tier of 1.90 — the lowest in the surveyed set. Industries are ranked by mean recommended browser-based retrieval tier.