Overview
JavaScript is Mexico's most common secondary barrier
| Value | |
|---|---|
| WAF | 92.5% |
| Antibot | 20.7% |
| CAPTCHA | 20.6% |
| JavaScript | 43.4% |
| Rate Limiting | 13.9% |
| TLS | 18.3% |
WAF covers 92.5% of Mexican sites. JavaScript (43.4%) leads secondary barriers. CAPTCHA (20.6%) is the lowest secondary barrier, sitting below Antibot (20.7%). Rate Limiting (13.9%) and TLS (18.3%) fill the lower positions.
Barrier Landscape
JavaScript tops Mexico's active secondary defences
| Value | |
|---|---|
| WAF | 92.5% |
| Antibot | 20.7% |
| CAPTCHA | 20.6% |
| JavaScript | 43.4% |
| Rate Limiting | 13.9% |
| TLS | 18.3% |
| Has robots.txt | 77.9% |
| Blocks AI crawlers | 13.4% |
WAF at 92.5% is near-universal. JavaScript (43.4%) leads secondary barriers by a wide margin. CAPTCHA (20.6%) is the lowest secondary barrier — below Antibot (20.7%). Rate Limiting (13.9%) and TLS (18.3%) follow.
Most Mexican sites carry at least two barriers
| % of sites | |
|---|---|
| 0 barriers | 5% |
| 1 barrier | 28.3% |
| 2 barriers | 36% |
| 3 barriers | 18.2% |
| 4 barriers | 8.9% |
| 5 barriers | 3.2% |
| 6 barriers | 0.5% |
36.0% of Mexican sites carry exactly two barriers — the most common count. Sites at three or more total 30.8%, including 8.9% at four. Just 5.0% have no barriers at all.
Most Mexican sites sit in the Easy access band
| Mean recommended tier | |
|---|---|
| All sites | 1.7 |
The browser median is Tier 2 — JavaScript rendering. Tier 1 (plain HTTP) covers 46.8% — close to half. Tiers 3–5 account for 13.2% of sites.
Most Mexican sites are Simple or Easy to access
| % of sites | |
|---|---|
| 1 - Simple | 46.8% |
| 2 - Easy | 40% |
| 3 - Moderate | 10.6% |
| 4 - Complex | 1.9% |
| 5 - Advanced | 0.7% |
Tier 1 (46.8%) and Tier 2 (40.0%) together account for 86.8% of Mexican sites. Tier 3 leads the difficult tail at 10.6%. Tiers 4 and 5 together add 2.6%.
robots.txt
AI crawlers top Mexico's robots.txt bot categories
| % of sites | |
|---|---|
| AI crawlers | 15.5% |
| SEO / search indexing | 11.3% |
| SEO audit tools | 12.8% |
| Commercial scrapers | 7.4% |
| Social / content scrapers | 6.1% |
| Archive / research crawlers | 4.5% |
| Monitoring / uptime bots | 0.5% |
AI crawlers are named in robots.txt by 15.5% of all Mexican sites. SEO audit tools appear on 12.8% and SEO/search indexers on 11.3%. Commercial scrapers reach 7.4%, social scrapers 6.1%, and archive crawlers 4.5%.
Uniform rules are Mexico's most common robots.txt posture
| % of sites | |
|---|---|
| Uniform per-agent rules (no path mixing) | 41.9% |
| Path-level mixed rules for same agent | 31.1% |
| Allow-only (no disallow rules) | 2.3% |
| Mixed AI posture (block some, allow others) | 2.5% |
| Blanket Disallow: / | 2.5% |
Uniform per-agent rules cover 41.9% of all Mexican sites. Path-level mixed rules apply to 31.1%. Mixed AI posture and blanket Disallow: / are level at 2.5% each.
GPTBot faces the most Disallow rules in Mexico's robots.txt
| Disallowed | Allowed (whitelisted) | |
|---|---|---|
| GPTBot | 11% | 2.4% |
| CCBot | 8.1% | 1.1% |
| ClaudeBot | 8.1% | 1.1% |
| Google-Extended | 7.8% | 1.7% |
| Bytespider | 6.3% | 0.3% |
| meta-externalagent | 2.9% | 0.4% |
| ChatGPT-User | 5.8% | 2.7% |
| PerplexityBot | 5.8% | 2.1% |
| Applebot-Extended | 4.5% | 0.4% |
| OAI-SearchBot | 4.1% | 2.1% |
| anthropic-ai | 5% | 0.7% |
GPTBot appears in Disallow rules on 11.0% of all Mexican sites. CCBot and ClaudeBot are equally blocked at 8.1% each. ChatGPT-User holds the highest Allow rate at 2.7%.
Four Mexican sectors all name at least one bot to restrict
| % of sites | |
|---|---|
| Computer & Video Games | 100% |
| Design | 100% |
| Retail | 100% |
| Visual Art | 100% |
| Adult | 90% |
| Aerospace & Defence | 90% |
| Beauty & Cosmetics | 90% |
| Civil Engineering | 90% |
| Comics & Animation | 90% |
| Consumer Electronics | 90% |
| Entertainment | 90% |
| General | 90% |
| Healthcare | 90% |
| Insurance | 90% |
| Machinery | 90% |
Computer & Video Games, Design, Retail, and Visual Art all reach 100%: every site in each sector names at least one bot in a Disallow rule. Adult, Aerospace & Defence, Beauty & Cosmetics, Civil Engineering, Comics & Animation, Consumer Electronics, Entertainment, General, Healthcare, Insurance, and Machinery follow at 90%.
Mexico's Photography, Publishing, Retail, and Science sectors lead in explicitly allowing crawlers
| % of sites | |
|---|---|
| Photography | 70% |
| Plastics & Polymers | 70% |
| Publishing | 70% |
| Retail | 70% |
| Science | 70% |
| Airlines | 60% |
| Architecture | 60% |
| Automotive | 60% |
| Beauty & Cosmetics | 60% |
| Facilities Services | 60% |
| Graphic Design | 60% |
| Investment | 60% |
| Oil & Gas | 60% |
| Public Relations | 60% |
| Renewable Energy | 60% |
Photography, Plastics & Polymers, Publishing, Retail, and Science each have 70% of their sites naming at least one bot in an Allow rule. Airlines, Architecture, Automotive, Beauty & Cosmetics, Facilities Services, Graphic Design, Investment, Oil & Gas, Public Relations, and Renewable Energy follow at 60%.
Industry Patterns
Hospitality and Jewellery are Mexico's most demanding sectors for crawlers
| WAF | Antibot | CAPTCHA | JavaScript | Rate Limit | TLS | robots.txt | Blocks AI | Tier | |
|---|---|---|---|---|---|---|---|---|---|
| Hospitality | 90% | 20% | 0% | 60% | 30% | 40% | 60% | 20% | Tier 3 |
| Jewellery & Luxury | 100% | 10% | 0% | 40% | 40% | 40% | 50% | 0% | Tier 3 |
| Real Estate | 100% | 10% | 10% | 70% | 40% | 30% | 50% | 20% | Tier 3 |
| Equipment & Supplies | 90% | 40% | 10% | 60% | 30% | 10% | 60% | 10% | Tier 3 |
| Medical Devices | 90% | 10% | 0% | 60% | 20% | 0% | 80% | 0% | Tier 3 |
| Automotive | 100% | 20% | 20% | 40% | 30% | 30% | 70% | 20% | Tier 2 |
| Furniture | 100% | 40% | 10% | 20% | 10% | 50% | 60% | 10% | Tier 2 |
| Restaurants | 90% | 30% | 30% | 60% | 50% | 30% | 50% | 10% | Tier 2 |
| Computer Hardware | 100% | 30% | 10% | 30% | 30% | 30% | 80% | 10% | Tier 2 |
| Retail | 100% | 10% | 0% | 60% | 20% | 20% | 100% | 40% | Tier 2 |
| Beauty & Cosmetics | 100% | 50% | 40% | 30% | 40% | 20% | 90% | 30% | Tier 2 |
| Consumer Electronics | 80% | 10% | 10% | 40% | 10% | 40% | 90% | 20% | Tier 2 |
| Human Resources | 100% | 60% | 60% | 50% | 40% | 20% | 70% | 10% | Tier 2 |
| Sporting Goods | 90% | 20% | 30% | 40% | 40% | 20% | 60% | 0% | Tier 2 |
| Apparel & Fashion | 100% | 20% | 30% | 70% | 30% | 30% | 80% | 10% | Tier 2 |
Hospitality, Jewellery & Luxury, and Real Estate lead at mean 2.70. Equipment & Supplies follows at 2.60. Medical Devices, Beauty & Cosmetics, Consumer Electronics, Human Resources, and Sporting Goods all share a mean of 2.20.
Manufacturing, Maritime, and Oil and Gas are Mexico's most accessible sectors
| WAF | Antibot | CAPTCHA | JavaScript | Rate Limit | TLS | robots.txt | Blocks AI | Tier | |
|---|---|---|---|---|---|---|---|---|---|
| Customer Services | 100% | 20% | 30% | 50% | 30% | 30% | 60% | 0% | Tier 2 |
| Crafts | 100% | 30% | 30% | 50% | 20% | 10% | 80% | 20% | Tier 2 |
| Design | 100% | 20% | 30% | 40% | 20% | 30% | 100% | 30% | Tier 2 |
| Insurance | 80% | 10% | 10% | 30% | 0% | 10% | 90% | 10% | Tier 2 |
| Mail & Delivery | 70% | 40% | 10% | 10% | 0% | 20% | 80% | 0% | Tier 2 |
| Public Safety | 100% | 30% | 40% | 70% | 20% | 40% | 90% | 10% | Tier 2 |
| Reference | 80% | 50% | 20% | 30% | 10% | 10% | 60% | 40% | Tier 2 |
| Telecom | 80% | 30% | 20% | 60% | 10% | 20% | 90% | 20% | Tier 2 |
| Transport & Logistics | 70% | 50% | 10% | 20% | 0% | 20% | 80% | 0% | Tier 2 |
| Wholesalers | 80% | 20% | 20% | 30% | 30% | 30% | 70% | 10% | Tier 2 |
| Food & Beverages | 100% | 50% | 20% | 60% | 40% | 0% | 60% | 20% | Tier 2 |
| Import & Export | 100% | 30% | 40% | 50% | 0% | 30% | 80% | 10% | Tier 2 |
| Manufacturing | 80% | 10% | 10% | 30% | 0% | 40% | 70% | 0% | Tier 2 |
| Maritime | 90% | 10% | 0% | 60% | 20% | 30% | 80% | 10% | Tier 2 |
| Oil & Gas | 70% | 10% | 20% | 40% | 0% | 10% | 70% | 0% | Tier 2 |
Import & Export, Manufacturing, Maritime, and Oil & Gas all reach a mean retrieval tier of 1.90 — the lowest across Mexican sectors in this group. Food & Beverages sits just above at 1.95.