Overview
JavaScript is India's most common secondary barrier
| Value | |
|---|---|
| WAF | 92.7% |
| Antibot | 21.7% |
| CAPTCHA | 22.7% |
| JavaScript | 42.2% |
| Rate Limiting | 13.2% |
| TLS | 18.9% |
WAF covers 92.7% of sites. Among secondary barriers, JavaScript leads at 42.2%. Antibot at 21.7% and TLS at 18.9% are both elevated.
Barrier Landscape
JavaScript tops India's active secondary defences
| Value | |
|---|---|
| WAF | 92.7% |
| Antibot | 21.7% |
| CAPTCHA | 22.7% |
| JavaScript | 42.2% |
| Rate Limiting | 13.2% |
| TLS | 18.9% |
| Has robots.txt | 79.7% |
| Blocks AI crawlers | 13.8% |
WAF dominates at 92.7%. JavaScript is the top secondary barrier at 42.2%. CAPTCHA follows at 22.7%, Antibot at 21.7%, TLS at 18.9%, and Rate Limiting at 13.2%.
Two-barrier sites narrowly outnumber those carrying three or more
| % of sites | |
|---|---|
| 0 barriers | 4.8% |
| 1 barrier | 29.8% |
| 2 barriers | 32.9% |
| 3 barriers | 18.7% |
| 4 barriers | 9.7% |
| 5 barriers | 4% |
| 6 barriers | 0.2% |
Two barriers is the most common configuration at 32.9% of sites. Three or more barriers appear on 32.6% — almost the same share. Barrier-free sites are 4.8%, and single-barrier sites account for 29.8%.
Most India sites are Simple to access
| Mean recommended tier | |
|---|---|
| All sites | 1.7 |
The mean browser tier of 1.71 across 1,111 sites reflects a large Tier 1 cohort. The median of 2 means fewer than half of sites are resolved at the simplest level, with Tier 2 the most frequent single recommendation.
More than eight in ten sites are Simple or Easy to access
| % of sites | |
|---|---|
| 1 - Simple | 46.5% |
| 2 - Easy | 40% |
| 3 - Moderate | 10.6% |
| 4 - Complex | 1.9% |
| 5 - Advanced | 1% |
Tier 1 applies to 46.5% of sites and Tier 2 to exactly 40.0%, totalling 86.5%. Tier 3 adds 10.6%. Tiers 4 and 5 remain marginal at 1.9% and 1.0% respectively.
robots.txt
AI crawlers lead by a narrow margin over SEO audit tools
| % of sites | |
|---|---|
| AI crawlers | 15.4% |
| SEO / search indexing | 12% |
| SEO audit tools | 14.7% |
| Commercial scrapers | 7.8% |
| Social / content scrapers | 6.1% |
| Archive / research crawlers | 4.2% |
| Monitoring / uptime bots | 0.5% |
AI crawlers are named in 15.4% of all 1,111 India sites. SEO audit tools follow at 14.7% — the smallest gap between the top two categories in this country group. SEO/search indexing bots reach 12.0%.
Uniform rules are India's most common robots.txt posture
| % of sites | |
|---|---|
| Uniform per-agent rules (no path mixing) | 41.9% |
| Path-level mixed rules for same agent | 32% |
| Allow-only (no disallow rules) | 2.9% |
| Mixed AI posture (block some, allow others) | 2.2% |
| Blanket Disallow: / | 2.6% |
Uniform per-agent rules apply on 41.9% of all sites, with path-level mixed rules on 32.0%. The allow-only posture — sites with no Disallow rules at all — reaches 2.9%. Sites with no robots.txt account for 20.3%.
GPTBot faces the most Disallow rules in India's robots.txt
| Disallowed | Allowed (whitelisted) | |
|---|---|---|
| GPTBot | 11.3% | 2.2% |
| CCBot | 8.7% | 1.1% |
| ClaudeBot | 8.1% | 1% |
| Google-Extended | 8.3% | 1.5% |
| Bytespider | 7% | 0.3% |
| meta-externalagent | 3.2% | 0.1% |
| ChatGPT-User | 6.1% | 2.3% |
| PerplexityBot | 6% | 2.1% |
| Applebot-Extended | 4.9% | 0.2% |
| OAI-SearchBot | 4.2% | 1.6% |
| anthropic-ai | 5.1% | 0.6% |
11.3% of all India sites carry a Disallow for GPTBot. CCBot and Google-Extended follow at 8.7% and 8.3%. ChatGPT-User leads explicit Allow rules at 2.3%.
Telecom joins six sectors at full named-bot Disallow coverage
| % of sites | |
|---|---|
| Computer & Video Games | 100% |
| Consumer Electronics | 100% |
| Design | 100% |
| Retail | 100% |
| Telecom | 100% |
| Visual Art | 100% |
| Adult | 90% |
| Beauty & Cosmetics | 90% |
| Civil Engineering | 90% |
| Comics & Animation | 90% |
| Entertainment | 90% |
| General | 90% |
| Healthcare | 90% |
| Machinery | 90% |
| Mass Media | 90% |
Computer & Video Games, Consumer Electronics, Design, Retail, Telecom, and Visual Art each show every measured site naming at least one bot to restrict. Telecom at 100% is distinctive. Nine further sectors sit at 90%.
India's Airlines sites lead in explicitly allowing crawlers
| % of sites | |
|---|---|
| Airlines | 80% |
| Photography | 70% |
| Plastics & Polymers | 70% |
| Publishing | 70% |
| Retail | 70% |
| Science | 70% |
| Telecom | 70% |
| Architecture | 60% |
| Automotive | 60% |
| Facilities Services | 60% |
| Graphic Design | 60% |
| Public Policy | 60% |
| Real Estate | 60% |
| Software & Dev | 60% |
| Sports | 60% |
Airlines is the top sector for explicit Allow rules at 80% of sites. Photography, Plastics & Polymers, Publishing, Retail, Science, and Telecom all follow at 70%.
Industry Patterns
Real Estate is the hardest sector
| WAF | Antibot | CAPTCHA | JavaScript | Rate Limit | TLS | robots.txt | Blocks AI | Tier | |
|---|---|---|---|---|---|---|---|---|---|
| Real Estate | 90% | 20% | 20% | 50% | 40% | 20% | 70% | 30% | Tier 3 |
| Hospitality | 90% | 30% | 0% | 60% | 20% | 40% | 60% | 10% | Tier 3 |
| Medical Devices | 100% | 20% | 10% | 60% | 20% | 0% | 90% | 0% | Tier 3 |
| Equipment & Supplies | 90% | 30% | 20% | 50% | 20% | 10% | 70% | 20% | Tier 3 |
| Furniture | 100% | 40% | 40% | 50% | 20% | 30% | 70% | 10% | Tier 3 |
| Apparel & Fashion | 100% | 10% | 20% | 70% | 30% | 10% | 80% | 10% | Tier 2 |
| Retail | 100% | 10% | 0% | 70% | 10% | 10% | 100% | 50% | Tier 2 |
| Human Resources | 100% | 80% | 60% | 50% | 40% | 20% | 70% | 10% | Tier 2 |
| Jewellery & Luxury | 100% | 30% | 0% | 50% | 40% | 50% | 70% | 10% | Tier 2 |
| Computer Hardware | 90% | 20% | 0% | 30% | 20% | 20% | 70% | 10% | Tier 2 |
| Customer Services | 100% | 20% | 40% | 60% | 40% | 10% | 50% | 0% | Tier 2 |
| Consumer Electronics | 80% | 10% | 10% | 40% | 10% | 50% | 100% | 20% | Tier 2 |
| Insurance | 80% | 10% | 0% | 30% | 10% | 30% | 70% | 0% | Tier 2 |
| Oil & Gas | 80% | 10% | 0% | 30% | 0% | 20% | 70% | 10% | Tier 2 |
| Telecom | 90% | 10% | 20% | 70% | 0% | 30% | 100% | 20% | Tier 2 |
Real Estate is the hardest sector at a mean recommended tier of 2.70. Hospitality and Medical Devices both sit at 2.60. The five hardest sectors all carry a tier median of 3, signalling Moderate complexity at the top.
Banking and Crafts share the most accessible floor
| WAF | Antibot | CAPTCHA | JavaScript | Rate Limit | TLS | robots.txt | Blocks AI | Tier | |
|---|---|---|---|---|---|---|---|---|---|
| Wine & Spirits | 100% | 60% | 10% | 50% | 30% | 10% | 90% | 30% | Tier 2 |
| Food & Beverages | 100% | 40% | 30% | 60% | 30% | 0% | 80% | 40% | Tier 2 |
| Warehousing | 100% | 70% | 40% | 80% | 70% | 10% | 40% | 10% | Tier 2 |
| Beauty & Cosmetics | 80% | 20% | 30% | 20% | 30% | 20% | 90% | 10% | Tier 2 |
| Restaurants | 90% | 40% | 40% | 50% | 30% | 20% | 60% | 20% | Tier 2 |
| Design | 100% | 20% | 30% | 40% | 20% | 30% | 100% | 30% | Tier 2 |
| Mail & Delivery | 70% | 40% | 10% | 10% | 0% | 20% | 80% | 0% | Tier 2 |
| Public Safety | 100% | 30% | 50% | 70% | 20% | 40% | 90% | 20% | Tier 2 |
| Recruitment & Staffing | 100% | 50% | 60% | 20% | 10% | 30% | 70% | 20% | Tier 2 |
| Reference | 80% | 60% | 30% | 40% | 20% | 10% | 50% | 30% | Tier 2 |
| Sporting Goods | 80% | 10% | 20% | 40% | 20% | 30% | 60% | 0% | Tier 2 |
| Transport & Logistics | 70% | 40% | 10% | 10% | 0% | 20% | 80% | 0% | Tier 2 |
| TV & Film | 100% | 60% | 10% | 60% | 50% | 10% | 60% | 10% | Tier 2 |
| Banking | 100% | 40% | 10% | 20% | 10% | 10% | 60% | 0% | Tier 2 |
| Crafts | 100% | 30% | 30% | 50% | 20% | 10% | 80% | 20% | Tier 2 |
Banking and Crafts reach the floor of this sample at a mean recommended tier of 1.90. Both carry a tier median of 2. Most sectors in the bottom group cluster between 2.00 and 2.20, indicating limited spread.