Overview
JavaScript is Estonia's most elevated secondary barrier
| Value | |
|---|---|
| WAF | 93% |
| Antibot | 18.9% |
| CAPTCHA | 21.3% |
| JavaScript | 43.6% |
| Rate Limiting | 12.7% |
| TLS | 16.8% |
WAF adoption stands at 93.0%, in line with the global rate of 93.1%. JavaScript rendering is required at 43.6% of sites, above the global 41.3%. Antibot (18.9%), CAPTCHA (21.3%), rate limiting (12.7%), and TLS (16.8%) all sit near or below global averages.
Barrier Landscape
JavaScript tops Estonia's active secondary defences
| Value | |
|---|---|
| WAF | 93% |
| Antibot | 18.9% |
| CAPTCHA | 21.3% |
| JavaScript | 43.6% |
| Rate Limiting | 12.7% |
| TLS | 16.8% |
| Has robots.txt | 81.7% |
| Blocks AI crawlers | 13.9% |
WAF adoption is 93.0%. Among secondary barriers, JavaScript leads at 43.6%, above the global 41.3%. CAPTCHA is the next highest secondary barrier at 21.3%.
Most Estonian sites carry at least two barriers
| % of sites | |
|---|---|
| 0 barriers | 4.5% |
| 1 barrier | 30.7% |
| 2 barriers | 35.3% |
| 3 barriers | 17.7% |
| 4 barriers | 7.4% |
| 5 barriers | 3.9% |
| 6 barriers | 0.5% |
4.5% of Estonian sites carry no barriers. The modal count is two barriers, at 35.3% of sites. Sites with three or more barriers combined account for 29.5%.
Most Estonian sites are Easy to access
| Mean recommended tier | |
|---|---|
| All sites | 1.6 |
The median recommended retrieval tier is 2, meaning fewer than half of sites are accessible via plain HTTP. The mean tier is 1.64.
Nearly nine in ten Estonian sites are Simple or Easy to access
| % of sites | |
|---|---|
| 1 - Simple | 49.6% |
| 2 - Easy | 39.5% |
| 3 - Moderate | 8.8% |
| 4 - Complex | 1.7% |
| 5 - Advanced | 0.4% |
Tier 1 accounts for 49.6% of sites and Tier 2 for 39.5%, giving a combined Tier 1–2 rate of 89.1%. Tier 3 covers 8.8%, Tier 4 1.7%, and Tier 5 0.4%.
robots.txt
AI crawlers top Estonia's robots.txt bot categories
| % of sites | |
|---|---|
| AI crawlers | 15.7% |
| SEO / search indexing | 12.6% |
| SEO audit tools | 13.7% |
| Commercial scrapers | 8.1% |
| Social / content scrapers | 6.2% |
| Archive / research crawlers | 4.9% |
| Monitoring / uptime bots | 0.6% |
Across all 1,105 Estonian sites, 15.7% name at least one AI crawler. SEO audit tools are named by 13.7% and SEO / search indexing bots by 12.6%.
Uniform rules are Estonia's most common robots.txt posture
| % of sites | |
|---|---|
| Uniform per-agent rules (no path mixing) | 42.5% |
| Path-level mixed rules for same agent | 33.8% |
| Allow-only (no disallow rules) | 2.6% |
| Mixed AI posture (block some, allow others) | 2.4% |
| Blanket Disallow: / | 2.5% |
18.3% of Estonian sites have no robots.txt. Of those that do, uniform per-agent rules account for 42.5%, path-level mixed rules for 33.8%, blanket Disallow: / for 2.5%, and mixed-AI posture for 2.4%.
GPTBot faces the most Disallow rules in Estonia's robots.txt
| Disallowed | Allowed (whitelisted) | |
|---|---|---|
| GPTBot | 11.4% | 2.2% |
| CCBot | 8.7% | 0.8% |
| ClaudeBot | 8.1% | 0.7% |
| Google-Extended | 8.1% | 1.2% |
| Bytespider | 6.8% | 0.3% |
| meta-externalagent | 3.1% | 0.1% |
| ChatGPT-User | 6% | 2.4% |
| PerplexityBot | 5.9% | 1.8% |
| Applebot-Extended | 4.8% | 0.1% |
| OAI-SearchBot | 4% | 2% |
| anthropic-ai | 5.1% | 0.5% |
GPTBot appears in a Disallow rule on 11.4% of all Estonian sites. ChatGPT-User is the most explicitly allowed, appearing in an Allow rule on 2.4% of all sites.
Six Estonian sectors all name at least one bot to restrict
| % of sites | |
|---|---|
| Design | 100% |
| Entertainment | 100% |
| Newspapers | 100% |
| Outsourcing | 100% |
| Plastics & Polymers | 100% |
| Visual Art | 100% |
| Adult | 90% |
| Automotive | 90% |
| Beauty & Cosmetics | 90% |
| Ceramics & Pottery | 90% |
| Chemicals | 90% |
| Comics & Animation | 90% |
| Computer & Video Games | 90% |
| Consulting | 90% |
| Facilities Services | 90% |
Design, Entertainment, Newspapers, Outsourcing, Plastics & Polymers, and Visual Art all reach 100% — every site in each sector names at least one bot to restrict in a Disallow rule.
Estonia's Airlines and Automotive sectors lead in allowing crawlers
| % of sites | |
|---|---|
| Airlines | 80% |
| Automotive | 80% |
| Investment | 70% |
| Photography | 70% |
| Publishing | 70% |
| Science | 70% |
| Travel & Tourism | 70% |
| Architecture | 60% |
| Ceramics & Pottery | 60% |
| Customer Services | 60% |
| Graphic Design | 60% |
| Import & Export | 60% |
| Non-Profit | 60% |
| Pharmaceuticals | 60% |
| Real Estate | 60% |
Airlines and Automotive are the top-ranked sectors, with 80.0% of sites in each naming at least one bot to allow in an Allow rule.
Industry Patterns
Hospitality and Jewellery & Luxury are Estonia's hardest sectors to access programmatically
| WAF | Antibot | CAPTCHA | JavaScript | Rate Limit | TLS | robots.txt | Blocks AI | Tier | |
|---|---|---|---|---|---|---|---|---|---|
| Hospitality | 90% | 20% | 0% | 60% | 30% | 40% | 60% | 10% | Tier 3 |
| Jewellery & Luxury | 100% | 20% | 20% | 30% | 40% | 60% | 60% | 0% | Tier 3 |
| Real Estate | 100% | 10% | 10% | 40% | 40% | 20% | 70% | 20% | Tier 2 |
| Computer Hardware | 90% | 10% | 0% | 20% | 20% | 20% | 80% | 10% | Tier 2 |
| Beauty & Cosmetics | 90% | 30% | 20% | 50% | 20% | 30% | 90% | 20% | Tier 2 |
| Apparel & Fashion | 90% | 10% | 20% | 60% | 20% | 30% | 90% | 10% | Tier 2 |
| Retail | 100% | 0% | 0% | 60% | 30% | 20% | 90% | 40% | Tier 2 |
| Consumer Electronics | 70% | 10% | 0% | 40% | 10% | 40% | 80% | 10% | Tier 2 |
| Equipment & Supplies | 90% | 30% | 20% | 60% | 20% | 10% | 70% | 20% | Tier 2 |
| Human Resources | 100% | 70% | 60% | 50% | 40% | 20% | 80% | 10% | Tier 2 |
| Restaurants | 90% | 40% | 30% | 60% | 40% | 20% | 60% | 10% | Tier 2 |
| Furniture | 90% | 50% | 30% | 50% | 30% | 20% | 60% | 10% | Tier 2 |
| Textiles & Nonwovens | 100% | 0% | 20% | 50% | 10% | 20% | 80% | 10% | Tier 2 |
| Food & Beverages | 100% | 40% | 20% | 50% | 30% | 0% | 70% | 20% | Tier 2 |
| Design | 100% | 20% | 30% | 40% | 20% | 30% | 100% | 30% | Tier 2 |
Hospitality and Jewellery & Luxury share the highest mean recommended tier at 2.70. A mid-table cluster of sectors — including Real Estate, Computer Hardware, and Beauty & Cosmetics — sits in the 2.40–2.60 range.
Seven Estonian sectors share the most accessible floor
| WAF | Antibot | CAPTCHA | JavaScript | Rate Limit | TLS | robots.txt | Blocks AI | Tier | |
|---|---|---|---|---|---|---|---|---|---|
| Machinery | 100% | 0% | 10% | 60% | 0% | 0% | 100% | 10% | Tier 2 |
| Mail & Delivery | 70% | 40% | 10% | 10% | 0% | 20% | 80% | 0% | Tier 2 |
| Maritime | 90% | 10% | 0% | 50% | 10% | 30% | 90% | 10% | Tier 2 |
| Medical Devices | 100% | 10% | 10% | 60% | 20% | 0% | 90% | 0% | Tier 2 |
| Public Safety | 100% | 30% | 40% | 60% | 20% | 20% | 90% | 20% | Tier 2 |
| Sporting Goods | 80% | 10% | 20% | 20% | 30% | 30% | 70% | 0% | Tier 2 |
| Transport & Logistics | 70% | 40% | 10% | 10% | 0% | 20% | 80% | 0% | Tier 2 |
| Wine & Spirits | 100% | 60% | 20% | 70% | 40% | 0% | 80% | 40% | Tier 2 |
| Adult | 100% | 40% | 30% | 80% | 10% | 20% | 90% | 10% | Tier 2 |
| Automotive | 100% | 20% | 0% | 20% | 30% | 30% | 90% | 10% | Tier 2 |
| Computers & Electronics | 80% | 10% | 30% | 30% | 0% | 20% | 60% | 0% | Tier 2 |
| Crafts | 100% | 30% | 40% | 50% | 20% | 10% | 80% | 20% | Tier 2 |
| Manufacturing | 80% | 10% | 10% | 30% | 0% | 40% | 70% | 0% | Tier 2 |
| Music | 100% | 40% | 30% | 70% | 30% | 20% | 80% | 10% | Tier 2 |
| Recruitment & Staffing | 100% | 30% | 40% | 50% | 0% | 10% | 70% | 10% | Tier 2 |
Adult, Automotive, Computers & Electronics, Crafts, Manufacturing, Music, and Recruitment & Staffing all record a mean recommended tier of 1.90 — the lowest in the bottom-industries list.