Overview
Russian sites deploy Antibot less often than most countries
| Value | |
|---|---|
| WAF | 93.2% |
| Antibot | 17.2% |
| CAPTCHA | 23.6% |
| JavaScript | 40.9% |
| Rate Limiting | 12.2% |
| TLS | 15.6% |
WAF adoption stands at 93.2%, matching the global rate of 93.1%. Antibot (17.2%) is the clearest secondary-barrier departure, running 3.0 points below the global rate of 20.2%. CAPTCHA (23.6%) sits slightly above its global benchmark of 23.1%.
Barrier Landscape
JavaScript tops Russia's active secondary barriers
| Value | |
|---|---|
| WAF | 93.2% |
| Antibot | 17.2% |
| CAPTCHA | 23.6% |
| JavaScript | 40.9% |
| Rate Limiting | 12.2% |
| TLS | 15.6% |
| Has robots.txt | 83.3% |
| Blocks AI crawlers | 14% |
WAF is deployed by 93.2% of Russia sites. Among secondary barriers, JavaScript ranks first at 40.9%, followed by CAPTCHA at 23.6% and Antibot at 17.2%. TLS (15.6%) and Rate Limiting (12.2%) are both below their global benchmarks.
Most Russian sites carry two barriers
| % of sites | |
|---|---|
| 0 barriers | 4.6% |
| 1 barrier | 31.9% |
| 2 barriers | 34.8% |
| 3 barriers | 17.7% |
| 4 barriers | 7.4% |
| 5 barriers | 3.2% |
| 6 barriers | 0.4% |
4.6% of Russia sites carry no detected barriers. The modal count is two barriers, accounting for 34.8% of sites. Sites deploying three or more barriers total 28.7%.
More than half of Russian sites are Simple to access
| Mean recommended tier | |
|---|---|
| All sites | 1.6 |
The browser-based median tier is 1, meaning over 50% of Russia sites are accessible via plain HTTP. The mean tier is 1.58 across 1,110 Russia sites.
Nearly nine in ten Russian sites are Simple or Easy to access
| % of sites | |
|---|---|
| 1 - Simple | 55% |
| 2 - Easy | 34.9% |
| 3 - Moderate | 8.1% |
| 4 - Complex | 1.4% |
| 5 - Advanced | 0.6% |
Tier 1 and Tier 2 combined account for 89.9% of Russia sites (55.0% and 34.9% respectively). Tier 3 holds 8.1%, Tier 4 holds 1.4%, and Tier 5 holds 0.6%.
robots.txt
AI crawlers and SEO bots are widely named in Russia's robots.txt
| % of sites | |
|---|---|
| AI crawlers | 15.9% |
| SEO / search indexing | 14.6% |
| SEO audit tools | 13.1% |
| Commercial scrapers | 7.6% |
| Social / content scrapers | 6.4% |
| Archive / research crawlers | 5.8% |
| Monitoring / uptime bots | 0.3% |
Across all 1,110 Russia sites, 15.9% name at least one AI crawler and 14.6% name at least one SEO/search indexing bot in a robots.txt rule. SEO audit tools appear on 13.1% of sites.
Uniform rules are Russia's most common robots.txt posture
| % of sites | |
|---|---|
| Uniform per-agent rules (no path mixing) | 42.3% |
| Path-level mixed rules for same agent | 35.7% |
| Allow-only (no disallow rules) | 2.4% |
| Mixed AI posture (block some, allow others) | 2.2% |
| Blanket Disallow: / | 2.9% |
16.7% of Russia sites have no robots.txt. Among those that do, uniform per-agent rules account for 42.3% of all sites, path-level mixed rules for 35.7%, blanket Disallow: / for 2.9%, and mixed-AI posture for 2.2%.
GPTBot faces the most Disallow rules in Russia's robots.txt
| Disallowed | Allowed (whitelisted) | |
|---|---|---|
| GPTBot | 11.2% | 2.3% |
| CCBot | 8.9% | 1.2% |
| ClaudeBot | 7.9% | 1.1% |
| Google-Extended | 7.4% | 1.3% |
| Bytespider | 7.1% | 0.5% |
| meta-externalagent | 2.8% | — |
| ChatGPT-User | 4.8% | 2.3% |
| PerplexityBot | 5% | 1.5% |
| Applebot-Extended | 5.3% | 0.5% |
| OAI-SearchBot | 3.3% | 1.8% |
| anthropic-ai | 4.8% | 0.9% |
GPTBot is named in a Disallow rule on 11.2% of all Russia sites — the highest block rate of any AI agent. GPTBot and ChatGPT-User are tied as the most explicitly allowed agents, each on 2.3% of all sites.
Four Russian sectors all name at least one bot to restrict
| % of sites | |
|---|---|
| Design | 100% |
| Insurance | 100% |
| Machinery | 100% |
| Visual Art | 100% |
| Adult | 90% |
| Agriculture | 90% |
| Apparel & Fashion | 90% |
| Beauty & Cosmetics | 90% |
| Comics & Animation | 90% |
| Construction | 90% |
| Crafts | 90% |
| Entertainment | 90% |
| Food & Beverages | 90% |
| Furniture | 90% |
| General | 90% |
Four sectors — Design, Insurance, Machinery, and Visual Art — reach 100%, meaning every site in those sectors names at least one bot to restrict in a Disallow rule. Eleven further sectors reach 90%.
Russia's Automotive, Publishing and Science sites lead in allowing crawlers
| % of sites | |
|---|---|
| Automotive | 80% |
| Publishing | 80% |
| Science | 80% |
| Airlines | 70% |
| Investment | 70% |
| Market Research | 70% |
| Photography | 70% |
| Plastics & Polymers | 70% |
| Renewable Energy | 70% |
| Restaurants | 70% |
| Sporting Goods | 70% |
| Travel & Tourism | 70% |
| Apparel & Fashion | 60% |
| Architecture | 60% |
| Banking | 60% |
Automotive, Publishing, and Science each have 80% of their sites naming at least one bot to allow in an Allow rule — the highest rate in Russia. Nine further sectors reach 70%.
Industry Patterns
Real Estate is Russia's hardest sector to access programmatically
| WAF | Antibot | CAPTCHA | JavaScript | Rate Limit | TLS | robots.txt | Blocks AI | Tier | |
|---|---|---|---|---|---|---|---|---|---|
| Real Estate | 100% | 30% | 20% | 50% | 60% | 20% | 60% | 20% | Tier 3 |
| Computer Hardware | 90% | 10% | 0% | 20% | 20% | 30% | 80% | 0% | Tier 3 |
| Hospitality | 100% | 20% | 0% | 60% | 30% | 40% | 60% | 10% | Tier 3 |
| Beauty & Cosmetics | 90% | 30% | 20% | 50% | 30% | 20% | 90% | 20% | Tier 2 |
| Jewellery & Luxury | 90% | 40% | 10% | 50% | 30% | 50% | 70% | 10% | Tier 2 |
| Retail | 100% | 0% | 10% | 80% | 10% | 0% | 80% | 40% | Tier 2 |
| Apparel & Fashion | 90% | 20% | 20% | 50% | 30% | 60% | 90% | 10% | Tier 2 |
| Consumer Electronics | 80% | 0% | 0% | 40% | 10% | 30% | 80% | 10% | Tier 2 |
| Human Resources | 100% | 60% | 50% | 40% | 60% | 30% | 60% | 10% | Tier 2 |
| Import & Export | 90% | 30% | 30% | 30% | 0% | 30% | 70% | 10% | Tier 2 |
| Public Safety | 100% | 20% | 40% | 50% | 10% | 20% | 90% | 10% | Tier 2 |
| Transport & Logistics | 70% | 40% | 10% | 10% | 0% | 20% | 80% | 0% | Tier 2 |
| Airlines | 100% | 20% | 10% | 60% | 20% | 20% | 90% | 20% | Tier 2 |
| Design | 100% | 20% | 30% | 40% | 20% | 30% | 100% | 30% | Tier 2 |
| Mail & Delivery | 70% | 40% | 10% | 10% | 0% | 20% | 80% | 0% | Tier 2 |
Real Estate records the highest mean tier at 2.70, followed by Computer Hardware at 2.65 and Hospitality at 2.60. Beauty & Cosmetics (2.40) sits above a cluster of Jewellery & Luxury and Retail, which both share a mean of 2.30.
Five Russian sectors share the most accessible tier floor
| WAF | Antibot | CAPTCHA | JavaScript | Rate Limit | TLS | robots.txt | Blocks AI | Tier | |
|---|---|---|---|---|---|---|---|---|---|
| Maritime | 90% | 20% | 0% | 40% | 10% | 20% | 90% | 10% | Tier 2 |
| Sporting Goods | 90% | 20% | 20% | 30% | 10% | 50% | 90% | 0% | Tier 2 |
| Adult | 100% | 40% | 30% | 80% | 10% | 20% | 90% | 10% | Tier 2 |
| Equipment & Supplies | 80% | 0% | 10% | 50% | 20% | 20% | 70% | 10% | Tier 2 |
| Events | 90% | 0% | 0% | 90% | 40% | 10% | 50% | 0% | Tier 2 |
| Libraries | 100% | 0% | 10% | 50% | 0% | 0% | 90% | 10% | Tier 2 |
| Recruitment & Staffing | 100% | 40% | 70% | 20% | 10% | 20% | 80% | 20% | Tier 2 |
| Travel & Tourism | 100% | 20% | 10% | 70% | 20% | 20% | 70% | 30% | Tier 2 |
| Veterinary | 100% | 30% | 20% | 40% | 20% | 20% | 90% | 10% | Tier 2 |
| Visual Art | 100% | 30% | 20% | 30% | 0% | 20% | 100% | 20% | Tier 2 |
| Banking | 100% | 0% | 10% | 50% | 0% | 10% | 70% | 10% | Tier 2 |
| Manufacturing | 100% | 20% | 20% | 60% | 10% | 30% | 80% | 10% | Tier 2 |
| Reference | 80% | 50% | 20% | 40% | 20% | 10% | 60% | 30% | Tier 2 |
| Science | 100% | 20% | 10% | 40% | 30% | 10% | 80% | 10% | Tier 2 |
| Telecom | 90% | 10% | 10% | 50% | 10% | 10% | 90% | 10% | Tier 2 |
Banking, Manufacturing, Reference, Science, and Telecom all record a mean tier of 1.80 — the accessibility floor across Russia's bottom industries. Ten further sectors sit just above at 1.90.