Overview
Gibraltar sites carry barriers close to global norms
| Value | |
|---|---|
| WAF | 93.5% |
| Antibot | 18.2% |
| CAPTCHA | 23.8% |
| JavaScript | 44.5% |
| Rate Limiting | 14.6% |
| TLS | 16.2% |
WAF adoption stands at 93.5%, above the global 93.1%. JavaScript rendering applies to 44.5% of sites versus a global 41.3%. CAPTCHA (23.8%), antibot (18.2%), rate limiting (14.6%), and TLS (16.2%) all sit near global benchmarks.
Barrier Landscape
JavaScript tops Gibraltar's active secondary defences
| Value | |
|---|---|
| WAF | 93.5% |
| Antibot | 18.2% |
| CAPTCHA | 23.8% |
| JavaScript | 44.5% |
| Rate Limiting | 14.6% |
| TLS | 16.2% |
| Has robots.txt | 78.4% |
| Blocks AI crawlers | 13% |
WAF adoption is 93.5%. Among secondary barriers, JavaScript leads at 44.5%, above the global 41.3%. CAPTCHA is the second-highest secondary barrier at 23.8%.
Most Gibraltar sites carry at least two barriers
| % of sites | |
|---|---|
| 0 barriers | 4.5% |
| 1 barrier | 29.8% |
| 2 barriers | 34% |
| 3 barriers | 18.9% |
| 4 barriers | 8.2% |
| 5 barriers | 4.5% |
| 6 barriers | 0.2% |
4.5% of Gibraltar sites carry no barriers. The modal count is two barriers, at 34.0% of sites. Sites with three or more barriers account for 31.8% combined.
Most Gibraltar sites are Simple to access
| Mean recommended tier | |
|---|---|
| All sites | 1.6 |
The median recommended retrieval tier is 1, indicating that over half of sites are accessible via plain HTTP. The mean tier is 1.57.
Most Gibraltar sites are Simple or Easy to access
| % of sites | |
|---|---|
| 1 - Simple | 56.8% |
| 2 - Easy | 32.1% |
| 3 - Moderate | 8.6% |
| 4 - Complex | 2.1% |
| 5 - Advanced | 0.4% |
Tier 1 accounts for 56.8% of sites and Tier 2 for 32.1%, giving a combined Tier 1–2 rate of 88.9%. Tier 3 covers 8.6%, Tier 4 2.1%, and Tier 5 0.4%.
robots.txt
AI crawlers top Gibraltar's robots.txt bot categories
| % of sites | |
|---|---|
| AI crawlers | 14.5% |
| SEO / search indexing | 14.2% |
| SEO audit tools | 13.8% |
| Commercial scrapers | 7.1% |
| Social / content scrapers | 6.9% |
| Archive / research crawlers | 5.5% |
| Monitoring / uptime bots | 0.3% |
Across all 917 Gibraltar sites, 14.5% name at least one AI crawler and 14.2% name at least one SEO / search indexing bot. SEO audit tools are named by 13.8%.
Path-mixed rules are notably prevalent in Gibraltar's robots.txt files
| % of sites | |
|---|---|
| Uniform per-agent rules (no path mixing) | 37.7% |
| Path-level mixed rules for same agent | 34.8% |
| Allow-only (no disallow rules) | 2.8% |
| Mixed AI posture (block some, allow others) | 2.5% |
| Blanket Disallow: / | 2.9% |
21.6% of Gibraltar sites have no robots.txt. Among the rest, uniform per-agent rules account for 37.7%, path-level mixed rules for 34.8%, blanket Disallow: / for 2.9%, and mixed-AI posture for 2.5%.
Google-Extended faces the most Disallow rules in Gibraltar's robots.txt
| Disallowed | Allowed (whitelisted) | |
|---|---|---|
| GPTBot | 11% | 2.1% |
| CCBot | 7.4% | 1.1% |
| ClaudeBot | 8% | 1.1% |
| Google-Extended | 8.4% | 1.5% |
| Bytespider | 6.5% | 0.7% |
| meta-externalagent | 2.7% | 0.1% |
| ChatGPT-User | 6% | 2.5% |
| PerplexityBot | 6.2% | 2.1% |
| Applebot-Extended | 4.5% | 0.5% |
| OAI-SearchBot | 4.6% | 1.9% |
| anthropic-ai | 4.3% | 1% |
Google-Extended appears in a Disallow rule on 8.4% of all Gibraltar sites. ChatGPT-User is the most explicitly allowed, appearing in an Allow rule on 2.5% of all sites.
Four Gibraltar sectors all name at least one bot to restrict
| % of sites | |
|---|---|
| Design | 100% |
| Plastics & Polymers | 100% |
| Public Relations | 100% |
| Textiles & Nonwovens | 100% |
| Adult | 90% |
| Apparel & Fashion | 90% |
| Beauty & Cosmetics | 90% |
| Computer & Video Games | 90% |
| Entertainment | 90% |
| Food & Beverages | 90% |
| General | 90% |
| Mass Media | 90% |
| Museums | 90% |
| Newspapers | 90% |
| Online Services | 90% |
Design, Plastics & Polymers, Public Relations, and Textiles & Nonwovens all reach 100% — every site in each sector names at least one bot to restrict in a Disallow rule.
Gibraltar's Airlines sector leads in explicitly allowing crawlers
| % of sites | |
|---|---|
| Airlines | 80% |
| Food & Beverages | 70% |
| Manufacturing | 70% |
| Mental Health | 70% |
| Photography | 70% |
| Publishing | 70% |
| Sporting Goods | 70% |
| Wellness | 70% |
| Automotive | 60% |
| Biotech | 60% |
| Construction | 60% |
| Consulting | 60% |
| Education | 60% |
| Museums | 60% |
| Non-Profit | 60% |
Airlines is the top-ranked sector, with 80.0% of sites naming at least one bot to allow in an Allow rule. Food & Beverages, Manufacturing, Mental Health, Photography, Publishing, Sporting Goods, and Wellness follow at 70.0%.
Industry Patterns
Computer Hardware is Gibraltar's hardest sector to access programmatically
| WAF | Antibot | CAPTCHA | JavaScript | Rate Limit | TLS | robots.txt | Blocks AI | Tier | |
|---|---|---|---|---|---|---|---|---|---|
| Computer Hardware | 80% | 10% | 10% | 10% | 20% | 40% | 90% | 0% | Tier 3 |
| Consumer Electronics | 80% | 20% | 20% | 50% | 20% | 40% | 80% | 20% | Tier 2 |
| Human Resources | 100% | 60% | 50% | 50% | 50% | 20% | 80% | 0% | Tier 2 |
| Retail | 100% | 0% | 0% | 70% | 20% | 10% | 90% | 50% | Tier 2 |
| Sporting Goods | 90% | 20% | 20% | 10% | 50% | 40% | 80% | 0% | Tier 2 |
| Real Estate | 100% | 30% | 30% | 80% | 50% | 10% | 30% | 10% | Tier 2 |
| Apparel & Fashion | 100% | 0% | 20% | 40% | 20% | 50% | 90% | 10% | Tier 2 |
| Education | 100% | 20% | 10% | 20% | 10% | 10% | 70% | 10% | Tier 2 |
| Jewellery & Luxury | 90% | 0% | 10% | 50% | 30% | 50% | 80% | 0% | Tier 2 |
| Transport & Logistics | 80% | 40% | 10% | 40% | 0% | 20% | 90% | 0% | Tier 2 |
| Beauty & Cosmetics | 70% | 20% | 20% | 40% | 30% | 40% | 90% | 20% | Tier 2 |
| Mail & Delivery | 80% | 30% | 30% | 20% | 10% | 30% | 80% | 0% | Tier 2 |
| Adult | 100% | 30% | 40% | 80% | 20% | 20% | 90% | 10% | Tier 2 |
| Automotive | 100% | 20% | 0% | 40% | 40% | 30% | 80% | 20% | Tier 2 |
| Banking | 100% | 20% | 10% | 50% | 10% | 0% | 70% | 0% | Tier 2 |
Computer Hardware records the highest mean recommended tier at 3.05. Consumer Electronics, Human Resources, Retail, and Sporting Goods follow at 2.40.
Nine Gibraltar sectors share the most accessible floor
| WAF | Antibot | CAPTCHA | JavaScript | Rate Limit | TLS | robots.txt | Blocks AI | Tier | |
|---|---|---|---|---|---|---|---|---|---|
| Food & Beverages | 100% | 10% | 20% | 90% | 0% | 10% | 100% | 10% | Tier 2 |
| Hospitality | 90% | 20% | 10% | 60% | 10% | 20% | 70% | 10% | Tier 2 |
| Music | 100% | 40% | 30% | 70% | 30% | 20% | 80% | 10% | Tier 2 |
| Science | 100% | 30% | 10% | 50% | 30% | 10% | 80% | 10% | Tier 2 |
| Wine & Spirits | 100% | 40% | 20% | 50% | 30% | 10% | 80% | 20% | Tier 2 |
| Libraries | 100% | 0% | 11.1% | 55.6% | 0% | 22.2% | 77.8% | 11.1% | Tier 2 |
| Airlines | 90% | 10% | 10% | 60% | 20% | 30% | 90% | 40% | Tier 2 |
| Construction | 100% | 20% | 40% | 60% | 20% | 10% | 80% | 10% | Tier 2 |
| Design | 100% | 30% | 40% | 40% | 30% | 30% | 100% | 40% | Tier 2 |
| Finance | 80% | 30% | 10% | 20% | 20% | 10% | 60% | 20% | Tier 2 |
| Furniture | 90% | 40% | 30% | 50% | 20% | 10% | 50% | 0% | Tier 2 |
| Insurance | 100% | 50% | 30% | 50% | 40% | 0% | 50% | 0% | Tier 2 |
| Publishing | 90% | 30% | 30% | 50% | 20% | 0% | 90% | 60% | Tier 2 |
| Telecom | 90% | 0% | 10% | 70% | 0% | 20% | 100% | 20% | Tier 2 |
| Wholesalers | 100% | 20% | 40% | 90% | 40% | 20% | 80% | 20% | Tier 2 |
Airlines, Construction, Design, Finance, Furniture, Insurance, Publishing, Telecom, and Wholesalers all record a mean recommended tier of 1.80 — the lowest in the bottom-industries list for Gibraltar.