Overview
JavaScript is Oman's most common secondary barrier
| Value | |
|---|---|
| WAF | 92.3% |
| Antibot | 21.3% |
| CAPTCHA | 23.1% |
| JavaScript | 42.1% |
| Rate Limiting | 13.4% |
| TLS | 19.1% |
WAF is near-universal at 92.3%. Among secondary barriers, JavaScript is deployed by 42.1% of sites, followed by CAPTCHA at 23.1% and Antibot at 21.3%. TLS (19.1%) and Rate Limiting (13.4%) are less common.
Barrier Landscape
CAPTCHA and Antibot are closely matched behind JavaScript
| Value | |
|---|---|
| WAF | 92.3% |
| Antibot | 21.3% |
| CAPTCHA | 23.1% |
| JavaScript | 42.1% |
| Rate Limiting | 13.4% |
| TLS | 19.1% |
| Has robots.txt | 79.2% |
| Blocks AI crawlers | 12.6% |
WAF dominates at 92.3%. JavaScript is the leading secondary barrier at 42.1%. CAPTCHA (23.1%) and Antibot (21.3%) are within two percentage points of each other. Rate Limiting is the least-deployed secondary at 13.4%.
Two barriers is the single most common site configuration
| % of sites | |
|---|---|
| 0 barriers | 4.9% |
| 1 barrier | 29.8% |
| 2 barriers | 34% |
| 3 barriers | 17.4% |
| 4 barriers | 9% |
| 5 barriers | 4.7% |
| 6 barriers | 0.4% |
34.0% of Oman sites carry exactly two barriers — the modal count. Just 4.9% have none. Nearly a third of sites (31.5%) stack three or more barriers, indicating a substantial cohort with layered defences.
Most sites are Simple or Easy to access
| Mean recommended tier | |
|---|---|
| All sites | 1.6 |
The median recommended tier is 2 and the mean is 1.65. Tier 1 and Tier 2 together account for 88.8% of sites, meaning fewer than one in eight requires anything beyond standard browser-based retrieval.
Nearly nine in ten sites are Simple or Easy to access
| % of sites | |
|---|---|
| 1 - Simple | 49.3% |
| 2 - Easy | 39.5% |
| 3 - Moderate | 8.5% |
| 4 - Complex | 1.9% |
| 5 - Advanced | 0.8% |
49.3% of sites are Tier 1 (Simple) and 39.5% are Tier 2 (Easy). Tier 3 accounts for 8.5%, with Tier 4 (1.9%) and Tier 5 (0.8%) making up a small residual requiring advanced extraction.
robots.txt
AI crawlers are the most-cited bot category in robots.txt
| % of sites | |
|---|---|
| AI crawlers | 14.8% |
| SEO / search indexing | 12.9% |
| SEO audit tools | 13.7% |
| Commercial scrapers | 7% |
| Social / content scrapers | 6.4% |
| Archive / research crawlers | 5% |
| Monitoring / uptime bots | 0.5% |
Across all 1,092 Oman domains, 14.8% name an AI crawler in robots.txt — the highest category. SEO audit tools follow at 13.7% and SEO/search indexing bots at 12.9%. Monitoring bots are named by just 0.5%.
Uniform per-agent rules dominate the robots.txt posture distribution
| % of sites | |
|---|---|
| Uniform per-agent rules (no path mixing) | 42% |
| Path-level mixed rules for same agent | 31.9% |
| Allow-only (no disallow rules) | 2.5% |
| Mixed AI posture (block some, allow others) | 2.1% |
| Blanket Disallow: / | 2.7% |
42.0% of all sites use uniform per-agent rules with no path mixing. A further 31.9% use path-level mixed rules. 20.8% of sites publish no robots.txt at all. Blanket Disallow: / applies to 2.7%.
GPTBot is the most blocked AI agent
| Disallowed | Allowed (whitelisted) | |
|---|---|---|
| GPTBot | 10.6% | 1.9% |
| CCBot | 7.9% | 0.7% |
| ClaudeBot | 7.4% | 0.9% |
| Google-Extended | 7.7% | 1.3% |
| Bytespider | 6.3% | 0.4% |
| meta-externalagent | 2.7% | 0.1% |
| ChatGPT-User | 5.3% | 2.3% |
| PerplexityBot | 5.2% | 2% |
| Applebot-Extended | 4.2% | 0.3% |
| OAI-SearchBot | 3.6% | 1.9% |
| anthropic-ai | 4.6% | 0.7% |
10.6% of all Oman sites name GPTBot in a Disallow rule — the highest of any AI agent. CCBot (7.9%) and Google-Extended (7.7%) follow. ChatGPT-User is the most explicitly allowed at 2.3%.
Seven sectors have every site naming at least one bot to restrict
| % of sites | |
|---|---|
| Computer & Video Games | 100% |
| Design | 100% |
| Entertainment | 100% |
| Machinery | 100% |
| Printing | 100% |
| Retail | 100% |
| Warehousing | 100% |
| Adult | 90% |
| Comics & Animation | 90% |
| Consumer Electronics | 90% |
| Facilities Services | 90% |
| General | 90% |
| Mass Media | 90% |
| Medical Devices | 90% |
| Newspapers | 90% |
Computer & Video Games, Design, Entertainment, Machinery, Printing, Retail, and Warehousing all show 100% of sites naming at least one bot in a Disallow rule. Eight further sectors reach 90%.
Airlines, Market Research, and Publishing lead explicit bot Allow rules
| % of sites | |
|---|---|
| Airlines | 80% |
| Market Research | 80% |
| Publishing | 80% |
| Retail | 80% |
| Photography | 70% |
| Public Policy | 62.5% |
| Architecture | 60% |
| Construction | 60% |
| Investment | 60% |
| Plastics & Polymers | 60% |
| Real Estate | 60% |
| Renewable Energy | 60% |
| Science | 60% |
| Software & Dev | 60% |
| Sporting Goods | 60% |
80% of sites in Airlines, Market Research, and Publishing name at least one bot in an explicit Allow rule. Photography follows at 70%. Public Policy (62.5%) and six sectors at 60% round out the top tier.
Industry Patterns
Jewellery & Luxury is the hardest sector to access programmatically
| WAF | Antibot | CAPTCHA | JavaScript | Rate Limit | TLS | robots.txt | Blocks AI | Tier | |
|---|---|---|---|---|---|---|---|---|---|
| Jewellery & Luxury | 100% | 0% | 0% | 40% | 50% | 50% | 60% | 0% | Tier 3 |
| Real Estate | 100% | 30% | 30% | 60% | 50% | 10% | 60% | 30% | Tier 3 |
| Hospitality | 90% | 30% | 0% | 60% | 20% | 40% | 60% | 10% | Tier 3 |
| Human Resources | 100% | 70% | 60% | 50% | 50% | 30% | 70% | 10% | Tier 3 |
| Medical Devices | 90% | 20% | 10% | 50% | 30% | 0% | 90% | 10% | Tier 2 |
| Retail | 100% | 10% | 0% | 70% | 10% | 10% | 100% | 50% | Tier 2 |
| Apparel & Fashion | 90% | 10% | 20% | 50% | 20% | 30% | 80% | 10% | Tier 2 |
| Furniture | 100% | 40% | 30% | 50% | 20% | 20% | 70% | 0% | Tier 2 |
| Beauty & Cosmetics | 80% | 30% | 20% | 40% | 40% | 30% | 80% | 10% | Tier 2 |
| Computer Hardware | 90% | 10% | 0% | 20% | 10% | 40% | 80% | 0% | Tier 2 |
| Oil & Gas | 70% | 10% | 0% | 40% | 0% | 30% | 70% | 10% | Tier 2 |
| Restaurants | 90% | 40% | 20% | 50% | 30% | 20% | 70% | 10% | Tier 2 |
| Consumer Electronics | 80% | 10% | 10% | 30% | 10% | 30% | 90% | 10% | Tier 2 |
| Customer Services | 100% | 30% | 40% | 10% | 10% | 50% | 80% | 10% | Tier 2 |
| Design | 100% | 20% | 30% | 40% | 20% | 30% | 100% | 30% | Tier 2 |
Jewellery & Luxury has the highest mean recommended tier at 2.80 — with 100% WAF adoption and 50% Rate Limiting. Real Estate (2.70) and Hospitality (2.60) follow. Human Resources ranks fourth at 2.50 with 70% Antibot deployment.
Food & Beverages anchors the most accessible sector group
| WAF | Antibot | CAPTCHA | JavaScript | Rate Limit | TLS | robots.txt | Blocks AI | Tier | |
|---|---|---|---|---|---|---|---|---|---|
| Mail & Delivery | 70% | 40% | 10% | 10% | 0% | 20% | 80% | 0% | Tier 2 |
| Maritime | 90% | 10% | 10% | 40% | 10% | 30% | 90% | 10% | Tier 2 |
| Recruitment & Staffing | 90% | 40% | 50% | 30% | 0% | 20% | 60% | 20% | Tier 2 |
| Reference | 80% | 60% | 30% | 40% | 20% | 10% | 50% | 30% | Tier 2 |
| Renewable Energy | 80% | 10% | 10% | 60% | 10% | 40% | 80% | 0% | Tier 2 |
| Sporting Goods | 90% | 10% | 20% | 30% | 30% | 30% | 80% | 0% | Tier 2 |
| Transport & Logistics | 70% | 40% | 10% | 10% | 0% | 20% | 80% | 0% | Tier 2 |
| Food & Beverages | 100% | 50% | 30% | 70% | 50% | 0% | 80% | 40% | Tier 2 |
| Adult | 100% | 40% | 30% | 80% | 10% | 20% | 90% | 10% | Tier 2 |
| Automotive | 100% | 20% | 0% | 30% | 40% | 20% | 70% | 20% | Tier 2 |
| Banking | 100% | 30% | 10% | 20% | 20% | 10% | 80% | 0% | Tier 2 |
| Manufacturing | 80% | 10% | 10% | 30% | 0% | 40% | 70% | 0% | Tier 2 |
| Public Safety | 100% | 30% | 30% | 50% | 20% | 50% | 90% | 0% | Tier 2 |
| Visual Art | 100% | 30% | 30% | 30% | 0% | 20% | 100% | 20% | Tier 2 |
| Wholesalers | 80% | 10% | 20% | 50% | 20% | 30% | 70% | 10% | Tier 2 |
Among more accessible sectors, Food & Beverages records the lowest mean tier at 1.95, despite 100% WAF and 50% Rate Limiting. Seven sectors cluster at a mean tier of 1.90.