Overview
Dutch sites favour JavaScript over CAPTCHA as a secondary barrier
| Value | |
|---|---|
| WAF | 93.5% |
| Antibot | 20% |
| CAPTCHA | 21.9% |
| JavaScript | 43.1% |
| Rate Limiting | 12.6% |
| TLS | 18.4% |
WAF (93.5%) dominates the radar as the near-universal baseline. Among secondary barriers, JavaScript at 43.1% is above the global 41.3%, while CAPTCHA at 21.9% sits below the global 23.1% — Dutch sites lean toward rendering complexity rather than visible challenge friction.
Barrier Landscape
JavaScript stands out among Dutch secondary barriers
| Value | |
|---|---|
| WAF | 93.5% |
| Antibot | 20% |
| CAPTCHA | 21.9% |
| JavaScript | 43.1% |
| Rate Limiting | 12.6% |
| TLS | 18.4% |
| Has robots.txt | 81.5% |
| Blocks AI crawlers | 13.8% |
WAF at 93.5% matches the global benchmark. Among secondary barriers, JavaScript at 43.1% is the most elevated — 2 points above the global 41.3%. CAPTCHA (21.9%) and Antibot (20.0%) sit close together at the next level.
Most Dutch sites carry two or more barriers
| % of sites | |
|---|---|
| 0 barriers | 4.1% |
| 1 barrier | 31.6% |
| 2 barriers | 33.1% |
| 3 barriers | 18.2% |
| 4 barriers | 8% |
| 5 barriers | 4.7% |
| 6 barriers | 0.3% |
Only 4.1% of sites present no barriers. Two barriers is the modal outcome at 33.1%. Sites at four or more account for 13.0%, with five or more reaching 5.0%.
Most Dutch sites need Easy-tier access to retrieve content
| Mean recommended tier | |
|---|---|
| All sites | 1.6 |
A median of 2 means the typical Dutch site requires at least some JavaScript rendering. The 11.7% at Tier 3 or above reflects the heavier defensive tail in the distribution.
Most Dutch sites are Simple or Easy to access
| % of sites | |
|---|---|
| 1 - Simple | 49.2% |
| 2 - Easy | 39.1% |
| 3 - Moderate | 9.3% |
| 4 - Complex | 1.8% |
| 5 - Advanced | 0.6% |
Tier 1 (49.2%) and Tier 2 (39.1%) cover 88.3% of the sample. Tier 3 stands at 9.3%, Tier 4 at 1.8%, and Tier 5 at 0.6%.
robots.txt
AI crawlers top Dutch robots.txt bot categories
| % of sites | |
|---|---|
| AI crawlers | 15.2% |
| SEO / search indexing | 12.6% |
| SEO audit tools | 14% |
| Commercial scrapers | 8% |
| Social / content scrapers | 5.6% |
| Archive / research crawlers | 5.1% |
| Monitoring / uptime bots | 0.5% |
15.2% of all 1,111 Dutch sites name at least one AI crawler. SEO audit tools (14.0%) and SEO/search indexers (12.6%) follow. Commercial scrapers (8.0%) rank fourth. Monitoring bots (0.5%) are effectively absent.
Uniform rules are the Netherlands' most common robots.txt posture
| % of sites | |
|---|---|
| Uniform per-agent rules (no path mixing) | 43.1% |
| Path-level mixed rules for same agent | 32.8% |
| Allow-only (no disallow rules) | 2.6% |
| Mixed AI posture (block some, allow others) | 2.5% |
| Blanket Disallow: / | 2.8% |
18.5% of sites publish no robots.txt. Both extremes are elevated: Blanket Disallow: / (2.8%), Mixed-AI (2.5%), and Allow-only (2.6%). Uniform posture (43.1%) remains the dominant stance.
GPTBot faces the most Disallow rules in Dutch robots.txt files
| Disallowed | Allowed (whitelisted) | |
|---|---|---|
| GPTBot | 11.3% | 2.2% |
| CCBot | 8.6% | 1.1% |
| ClaudeBot | 8.3% | 1% |
| Google-Extended | 8% | 1.2% |
| Bytespider | 6.8% | 0.4% |
| meta-externalagent | 2.5% | — |
| ChatGPT-User | 5.9% | 2.4% |
| PerplexityBot | 5.8% | 1.9% |
| Applebot-Extended | 5% | 0.5% |
| OAI-SearchBot | 4.2% | 2.1% |
| anthropic-ai | 4.9% | 0.5% |
Values are percentages of all 1,111 Dutch sites. Bytespider's 6.8% block rate is low relative to other AI agents in this chart. OAI-SearchBot receives 2.1% explicit allows — the highest individual allow rate across all agents shown.
Medical Devices and Textiles sites all name a bot to restrict
| % of sites | |
|---|---|
| Computer & Video Games | 100% |
| Design | 100% |
| Entertainment | 100% |
| Medical Devices | 100% |
| Retail | 100% |
| Textiles & Nonwovens | 100% |
| Visual Art | 100% |
| Adult | 90% |
| Agriculture | 90% |
| Automotive | 90% |
| Civil Engineering | 90% |
| Comics & Animation | 90% |
| Food & Beverages | 90% |
| General | 90% |
| Machinery | 90% |
Medical Devices and Textiles & Nonwovens both reach 100% — every sampled site in each sector names at least one crawler in a Disallow rule. Both are Netherlands-specific appearances at the top. Civil Engineering at 90% is a further distinctive entry.
Dutch sites spread bot allowlisting evenly across sectors
| % of sites | |
|---|---|
| Airlines | 70% |
| Automotive | 70% |
| Non-Profit | 70% |
| Photography | 70% |
| Publishing | 70% |
| Restaurants | 70% |
| Retail | 70% |
| Science | 70% |
| Sporting Goods | 70% |
| Telecom | 70% |
| Travel & Tourism | 70% |
| Crafts | 60% |
| Farms & Ranches | 60% |
| Food & Beverages | 60% |
| Graphic Design | 60% |
No Dutch sector reaches an 80% allow rate — openness is distributed broadly rather than concentrated in a few enthusiastic sectors. Eleven sectors share exactly 70%: Airlines, Automotive, Non-Profit, Photography, Publishing, Restaurants, Retail, Science, Sporting Goods, Telecom, and Travel & Tourism.
Industry Patterns
Hospitality is the Netherlands' hardest sector to access programmatically
| WAF | Antibot | CAPTCHA | JavaScript | Rate Limit | TLS | robots.txt | Blocks AI | Tier | |
|---|---|---|---|---|---|---|---|---|---|
| Hospitality | 90% | 20% | 0% | 50% | 40% | 50% | 60% | 20% | Tier 3 |
| Jewellery & Luxury | 100% | 20% | 10% | 30% | 30% | 50% | 60% | 0% | Tier 3 |
| Real Estate | 100% | 30% | 30% | 60% | 50% | 10% | 60% | 20% | Tier 3 |
| Computer Hardware | 90% | 30% | 10% | 30% | 40% | 20% | 80% | 10% | Tier 3 |
| Automotive | 100% | 10% | 0% | 30% | 20% | 30% | 90% | 0% | Tier 2 |
| Beauty & Cosmetics | 90% | 20% | 20% | 60% | 30% | 20% | 80% | 10% | Tier 2 |
| Textiles & Nonwovens | 100% | 0% | 30% | 50% | 0% | 10% | 100% | 10% | Tier 2 |
| Warehousing | 100% | 60% | 40% | 60% | 40% | 10% | 70% | 0% | Tier 2 |
| Apparel & Fashion | 90% | 10% | 20% | 50% | 10% | 40% | 80% | 10% | Tier 2 |
| Consumer Electronics | 90% | 10% | 20% | 50% | 20% | 30% | 80% | 20% | Tier 2 |
| Customer Services | 100% | 20% | 30% | 60% | 30% | 20% | 70% | 0% | Tier 2 |
| Human Resources | 100% | 60% | 50% | 50% | 50% | 20% | 80% | 0% | Tier 2 |
| Medical Devices | 100% | 0% | 20% | 40% | 10% | 0% | 100% | 10% | Tier 2 |
| Restaurants | 90% | 30% | 20% | 70% | 50% | 30% | 80% | 10% | Tier 2 |
| Retail | 100% | 0% | 0% | 60% | 20% | 30% | 100% | 40% | Tier 2 |
Hospitality (2.90) leads, ahead of Jewellery & Luxury (2.80), Real Estate (2.80), and Computer Hardware (2.75). Warehousing (2.25) is a Netherlands-specific entry in the top 15.
Six Dutch sectors share the most accessible tier floor
| WAF | Antibot | CAPTCHA | JavaScript | Rate Limit | TLS | robots.txt | Blocks AI | Tier | |
|---|---|---|---|---|---|---|---|---|---|
| Crafts | 100% | 30% | 30% | 50% | 20% | 10% | 80% | 30% | Tier 2 |
| Maritime | 90% | 10% | 0% | 40% | 10% | 30% | 90% | 10% | Tier 2 |
| Public Safety | 100% | 30% | 40% | 60% | 20% | 40% | 90% | 10% | Tier 2 |
| Food & Beverages | 100% | 50% | 20% | 60% | 30% | 0% | 90% | 30% | Tier 2 |
| Design | 100% | 20% | 30% | 40% | 20% | 30% | 100% | 30% | Tier 2 |
| Equipment & Supplies | 90% | 50% | 40% | 60% | 40% | 0% | 50% | 0% | Tier 2 |
| Furniture | 100% | 20% | 20% | 70% | 30% | 10% | 60% | 10% | Tier 2 |
| Machinery | 90% | 0% | 10% | 60% | 0% | 0% | 90% | 10% | Tier 2 |
| Mail & Delivery | 70% | 40% | 10% | 10% | 0% | 20% | 80% | 0% | Tier 2 |
| Education | 100% | 20% | 10% | 30% | 10% | 10% | 70% | 10% | Tier 2 |
| Manufacturing | 80% | 10% | 10% | 30% | 0% | 40% | 70% | 0% | Tier 2 |
| Reference | 90% | 60% | 30% | 50% | 30% | 10% | 60% | 30% | Tier 2 |
| Sporting Goods | 80% | 10% | 30% | 20% | 30% | 30% | 70% | 10% | Tier 2 |
| Telecom | 90% | 0% | 10% | 60% | 0% | 30% | 90% | 20% | Tier 2 |
| Transport & Logistics | 70% | 40% | 10% | 20% | 0% | 20% | 80% | 0% | Tier 2 |
The floor sits at 1.90, shared by Education, Manufacturing, Reference, Sporting Goods, Telecom, and Transport & Logistics. Above that, a group at 2.00 includes Design, Equipment & Supplies, Furniture, Machinery, and Mail & Delivery.