Overview
JavaScript is United Arab Emirates's most common secondary barrier
| Value | |
|---|---|
| WAF | 92.4% |
| Antibot | 20.8% |
| CAPTCHA | 21% |
| JavaScript | 41.4% |
| Rate Limiting | 12.7% |
| TLS | 18.4% |
WAF covers 92.4% of 1,108 surveyed sites. JavaScript is the clear secondary leader at 41.4%. CAPTCHA (21.0%) and Antibot (20.8%) sit within 0.2 percentage points of each other — the tightest pair among the six barriers.
Barrier Landscape
JavaScript tops United Arab Emirates's active secondary defences
| Value | |
|---|---|
| WAF | 92.4% |
| Antibot | 20.8% |
| CAPTCHA | 21% |
| JavaScript | 41.4% |
| Rate Limiting | 12.7% |
| TLS | 18.4% |
| Has robots.txt | 79.6% |
| Blocks AI crawlers | 13.8% |
WAF leads at 92.4%. JavaScript (41.4%) is the dominant secondary barrier, running 20 points ahead of CAPTCHA (21.0%). TLS (18.4%) and Rate Limiting (12.7%) are the least-deployed of the six barriers.
Two-barrier sites are United Arab Emirates's most common configuration
| % of sites | |
|---|---|
| 0 barriers | 4.6% |
| 1 barrier | 31.1% |
| 2 barriers | 33.8% |
| 3 barriers | 18% |
| 4 barriers | 8.5% |
| 5 barriers | 3.7% |
| 6 barriers | 0.3% |
4.6% of sites present no detectable barriers. The modal configuration is exactly two barriers, covering 33.8% of sites. Sites stacking three or more barriers account for 30.5% of the surveyed domain set.
Most United Arab Emirates sites are Easy to access
| Mean recommended tier | |
|---|---|
| All sites | 1.7 |
Mean recommended tier is 1.68, median is 2. A median of 2 indicates fewer than half of sites resolve at Tier 1 (Simple). Tier 1 covers 48.2%; the remaining 51.8% require at least Easy-level access methods.
Nearly nine in ten sites are Simple or Easy to access
| % of sites | |
|---|---|
| 1 - Simple | 48.2% |
| 2 - Easy | 39.4% |
| 3 - Moderate | 9.7% |
| 4 - Complex | 2.1% |
| 5 - Advanced | 0.6% |
Tier 1 (Simple) covers 48.2% of sites; Tier 2 (Easy) adds 39.4%, bringing the combined share to 87.6%. Tier 3 (Moderate) accounts for 9.7%, Tier 4 for 2.1%, and Tier 5 for 0.6%.
robots.txt
AI crawlers edge out SEO audit tools as the most-cited bot category
| % of sites | |
|---|---|
| AI crawlers | 15.8% |
| SEO / search indexing | 12.9% |
| SEO audit tools | 14.3% |
| Commercial scrapers | 7.9% |
| Social / content scrapers | 6% |
| Archive / research crawlers | 4.5% |
| Monitoring / uptime bots | 0.5% |
AI crawlers are named in robots.txt by 15.8% of all 1,108 surveyed sites — the highest of any category. SEO audit tools follow at 14.3% and SEO / search indexing at 12.9%. All figures are percentages of the full surveyed set.
Uniform per-agent rules cover four in ten sites
| % of sites | |
|---|---|
| Uniform per-agent rules (no path mixing) | 41.1% |
| Path-level mixed rules for same agent | 33.4% |
| Allow-only (no disallow rules) | 2.6% |
| Mixed AI posture (block some, allow others) | 2.6% |
| Blanket Disallow: / | 2.4% |
20.4% of sites have no robots.txt. Among those that do, uniform per-agent rules are the most common posture at 41.1% of all sites. Path-level mixed rules apply to 33.4%, and blanket Disallow: / to 2.4%.
GPTBot draws the most blocks and the most explicit Allow rules
| Disallowed | Allowed (whitelisted) | |
|---|---|---|
| GPTBot | 11.5% | 2.9% |
| CCBot | 8.6% | 1.1% |
| ClaudeBot | 8.3% | 1.4% |
| Google-Extended | 8% | 1.5% |
| Bytespider | 6.6% | 0.4% |
| meta-externalagent | 2.9% | 0.3% |
| ChatGPT-User | 6% | 2.8% |
| PerplexityBot | 5.9% | 2.3% |
| Applebot-Extended | 4.7% | 0.4% |
| OAI-SearchBot | 4.2% | 2.1% |
| anthropic-ai | 5.1% | 0.8% |
GPTBot is named in a Disallow rule by 11.5% of all surveyed sites — the highest of any AI agent — and in an explicit Allow rule by 2.9%, also the highest. CCBot is the second most-blocked at 8.6%.
Seven UAE sectors all name at least one bot to restrict
| % of sites | |
|---|---|
| Computer & Video Games | 100% |
| Consumer Electronics | 100% |
| Design | 100% |
| Entertainment | 100% |
| Retail | 100% |
| Telecom | 100% |
| Visual Art | 100% |
| Adult | 90% |
| Beauty & Cosmetics | 90% |
| General | 90% |
| Healthcare | 90% |
| Machinery | 90% |
| Mass Media | 90% |
| Medical Devices | 90% |
| Newspapers | 90% |
Every site in seven sectors — including Computer & Video Games, Retail, and Telecom — names at least one bot to restrict in robots.txt. Eight additional sectors show a 90% rate, indicating near-universal named-bot restriction across a wide range of industries.
Airlines, Retail, and Telecom lead explicit bot Allow rules
| % of sites | |
|---|---|
| Airlines | 80% |
| Retail | 80% |
| Telecom | 80% |
| Photography | 70% |
| Publishing | 70% |
| Real Estate | 70% |
| Automotive | 60% |
| Graphic Design | 60% |
| Pharmaceuticals | 60% |
| Plastics & Polymers | 60% |
| Restaurants | 60% |
| Science | 60% |
| Software & Dev | 60% |
| Sports | 60% |
| Travel & Tourism | 60% |
Airlines, Retail, and Telecom each show an 80% explicit bot Allow rate — the highest in the country. Photography, Publishing, and Real Estate follow at 70%. Explicit Allow rules appear across a broad set of industry sectors.
Industry Patterns
Jewellery & Luxury is the hardest sector at Moderate mean difficulty
| WAF | Antibot | CAPTCHA | JavaScript | Rate Limit | TLS | robots.txt | Blocks AI | Tier | |
|---|---|---|---|---|---|---|---|---|---|
| Jewellery & Luxury | 100% | 20% | 10% | 30% | 50% | 40% | 70% | 0% | Tier 3 |
| Hospitality | 90% | 20% | 0% | 60% | 20% | 40% | 60% | 20% | Tier 3 |
| Real Estate | 100% | 30% | 30% | 50% | 30% | 10% | 70% | 40% | Tier 3 |
| Furniture | 100% | 40% | 20% | 30% | 20% | 40% | 70% | 10% | Tier 3 |
| Apparel & Fashion | 90% | 10% | 20% | 60% | 20% | 20% | 80% | 10% | Tier 2 |
| Computer Hardware | 90% | 10% | 0% | 20% | 20% | 20% | 80% | 10% | Tier 2 |
| Retail | 100% | 10% | 0% | 70% | 10% | 10% | 100% | 50% | Tier 2 |
| Beauty & Cosmetics | 80% | 40% | 20% | 20% | 40% | 40% | 90% | 10% | Tier 2 |
| Consumer Electronics | 80% | 10% | 20% | 60% | 10% | 50% | 100% | 10% | Tier 2 |
| Equipment & Supplies | 90% | 30% | 30% | 60% | 30% | 0% | 70% | 20% | Tier 2 |
| Human Resources | 100% | 70% | 60% | 60% | 40% | 20% | 70% | 10% | Tier 2 |
| Medical Devices | 100% | 10% | 10% | 50% | 10% | 0% | 100% | 0% | Tier 2 |
| Food & Beverages | 100% | 50% | 20% | 60% | 40% | 0% | 80% | 30% | Tier 2 |
| Public Safety | 100% | 30% | 40% | 60% | 20% | 40% | 90% | 10% | Tier 2 |
| Telecom | 90% | 10% | 10% | 70% | 0% | 30% | 100% | 20% | Tier 2 |
Jewellery & Luxury averages a mean recommended tier of 3.00 — the highest of any sector surveyed. Hospitality, Real Estate, and Retail all share a mean of 2.50. Industries are ranked by mean recommended browser-based retrieval tier.
Seven sectors share the most accessible floor
| WAF | Antibot | CAPTCHA | JavaScript | Rate Limit | TLS | robots.txt | Blocks AI | Tier | |
|---|---|---|---|---|---|---|---|---|---|
| Textiles & Nonwovens | 100% | 0% | 30% | 40% | 10% | 20% | 90% | 10% | Tier 2 |
| Wine & Spirits | 100% | 50% | 0% | 50% | 30% | 10% | 80% | 20% | Tier 2 |
| Banking | 90% | 30% | 10% | 30% | 10% | 20% | 70% | 10% | Tier 2 |
| Customer Services | 100% | 10% | 30% | 40% | 20% | 30% | 60% | 0% | Tier 2 |
| Design | 100% | 20% | 30% | 40% | 20% | 30% | 100% | 30% | Tier 2 |
| Mail & Delivery | 70% | 40% | 10% | 10% | 0% | 20% | 80% | 0% | Tier 2 |
| Maritime | 90% | 10% | 10% | 30% | 10% | 30% | 90% | 10% | Tier 2 |
| Transport & Logistics | 70% | 40% | 10% | 10% | 0% | 20% | 80% | 0% | Tier 2 |
| Adult | 100% | 40% | 30% | 80% | 10% | 20% | 90% | 10% | Tier 2 |
| Crafts | 100% | 20% | 30% | 50% | 10% | 10% | 80% | 20% | Tier 2 |
| Manufacturing | 80% | 10% | 10% | 30% | 0% | 40% | 70% | 0% | Tier 2 |
| Music | 100% | 40% | 30% | 70% | 30% | 20% | 80% | 10% | Tier 2 |
| Recruitment & Staffing | 100% | 40% | 50% | 30% | 0% | 20% | 80% | 20% | Tier 2 |
| Reference | 90% | 60% | 30% | 50% | 30% | 10% | 60% | 30% | Tier 2 |
| Restaurants | 90% | 50% | 30% | 60% | 40% | 20% | 80% | 20% | Tier 2 |
Adult, Crafts, Manufacturing, Music, Recruitment & Staffing, Reference, and Restaurants all average a mean recommended tier of 1.90 — the lowest in the surveyed set. Industries are ranked by mean recommended browser-based retrieval tier.