Overview
Luxembourg sites track global barrier rates closely
| Value | |
|---|---|
| WAF | 93.6% |
| Antibot | 19.7% |
| CAPTCHA | 21.8% |
| JavaScript | 42.8% |
| Rate Limiting | 12.4% |
| TLS | 17.1% |
WAF (93.6%) is marginally above the 93.1% global rate. JavaScript (42.8%) is slightly elevated. CAPTCHA (21.8%), antibot (19.7%), TLS (17.1%), and rate limiting (12.4%) are all within one percentage point of their global benchmarks.
Barrier Landscape
JavaScript tops Luxembourg's active secondary barriers
| Value | |
|---|---|
| WAF | 93.6% |
| Antibot | 19.7% |
| CAPTCHA | 21.8% |
| JavaScript | 42.8% |
| Rate Limiting | 12.4% |
| TLS | 17.1% |
| Has robots.txt | 81.1% |
| Blocks AI crawlers | 13.9% |
WAF adoption stands at 93.6%. Among secondary barriers, JavaScript leads at 42.8%, followed by CAPTCHA at 21.8%. Antibot (19.7%), TLS (17.1%), and rate limiting (12.4%) round out the picture.
Most Luxembourg sites carry two barriers
| % of sites | |
|---|---|
| 0 barriers | 4.2% |
| 1 barrier | 31.3% |
| 2 barriers | 34.3% |
| 3 barriers | 18% |
| 4 barriers | 7.8% |
| 5 barriers | 4% |
| 6 barriers | 0.4% |
4.2% of Luxembourg sites carry no barriers. The modal count is two, applying to 34.3% of sites. Three or more barriers account for 30.2% of the total.
Most Luxembourg sites are Simple to access
| Mean recommended tier | |
|---|---|
| All sites | 1.6 |
The median recommended retrieval tier is 1, meaning over half of sites are accessible without JavaScript rendering. The mean tier is 1.63.
Nearly nine in ten Luxembourg sites are Simple or Easy to access
| % of sites | |
|---|---|
| 1 - Simple | 51.4% |
| 2 - Easy | 37.1% |
| 3 - Moderate | 8.9% |
| 4 - Complex | 2.2% |
| 5 - Advanced | 0.5% |
Tier 1 and Tier 2 combined account for 88.5% of sites. Tier 3 applies to 8.9%, Tier 4 to 2.2%, and Tier 5 to 0.5%.
robots.txt
AI crawlers top Luxembourg's robots.txt bot categories
| % of sites | |
|---|---|
| AI crawlers | 15.9% |
| SEO / search indexing | 13.9% |
| SEO audit tools | 14% |
| Commercial scrapers | 8.1% |
| Social / content scrapers | 6.2% |
| Archive / research crawlers | 5.3% |
| Monitoring / uptime bots | 0.6% |
Across all 1,104 Luxembourg sites, 15.9% name at least one AI crawler. SEO audit tools follow at 14.0% and SEO/search indexing bots at 13.9%. Archive/research crawlers appear in 5.3% of files.
Uniform rules are Luxembourg's most common robots.txt posture
| % of sites | |
|---|---|
| Uniform per-agent rules (no path mixing) | 40.9% |
| Path-level mixed rules for same agent | 35.1% |
| Allow-only (no disallow rules) | 2.1% |
| Mixed AI posture (block some, allow others) | 2.3% |
| Blanket Disallow: / | 2.7% |
18.9% of Luxembourg sites have no robots.txt. Uniform per-agent rules lead at 40.9%, followed by path-level mixed rules at 35.1%. Blanket Disallow: / covers 2.7% and mixed-AI posture 2.3%.
GPTBot faces the most Disallow rules in Luxembourg's robots.txt
| Disallowed | Allowed (whitelisted) | |
|---|---|---|
| GPTBot | 11.1% | 2.4% |
| CCBot | 9% | 0.9% |
| ClaudeBot | 8.2% | 1.2% |
| Google-Extended | 7.7% | 1.1% |
| Bytespider | 6.8% | 0.5% |
| meta-externalagent | 3.1% | 0.2% |
| ChatGPT-User | 5.3% | 2.7% |
| PerplexityBot | 5.6% | 2% |
| Applebot-Extended | 5.1% | 0.5% |
| OAI-SearchBot | 4% | 1.9% |
| anthropic-ai | 4.7% | 0.7% |
GPTBot carries a Disallow rule on 11.1% of all 1,104 Luxembourg sites. ChatGPT-User appears in an Allow rule on 2.7% of sites.
Five Luxembourg sectors all name at least one bot to restrict
| % of sites | |
|---|---|
| Computer & Video Games | 100% |
| Design | 100% |
| Entertainment | 100% |
| Food & Beverages | 100% |
| Newspapers | 100% |
| Adult | 90% |
| Agriculture | 90% |
| Automotive | 90% |
| Beauty & Cosmetics | 90% |
| Civil Engineering | 90% |
| General | 90% |
| Machinery | 90% |
| Mass Media | 90% |
| Online Services | 90% |
| Outsourcing | 90% |
Computer & Video Games, Design, Entertainment, Food & Beverages, and Newspapers each reach 100%: every site in those sectors names at least one bot to restrict in a Disallow rule. Ten further sectors sit at 90%.
Luxembourg's Automotive sites lead in explicitly allowing crawlers
| % of sites | |
|---|---|
| Automotive | 80% |
| Airlines | 70% |
| Food & Beverages | 70% |
| Mental Health | 70% |
| Photography | 70% |
| Publishing | 70% |
| Real Estate | 70% |
| Retail | 70% |
| Science | 70% |
| Sports | 70% |
| Travel & Tourism | 70% |
| Education | 60% |
| Hospitality | 60% |
| Import & Export | 60% |
| Non-Profit | 60% |
Automotive leads by a clear margin: 80% of sites in this industry explicitly allow at least one named crawler, reflecting commercial demand for indexing and aggregation. Airlines follows at 70%.
Industry Patterns
Jewellery & Luxury and Real Estate are Luxembourg's hardest sectors to access
| WAF | Antibot | CAPTCHA | JavaScript | Rate Limit | TLS | robots.txt | Blocks AI | Tier | |
|---|---|---|---|---|---|---|---|---|---|
| Jewellery & Luxury | 100% | 20% | 20% | 30% | 40% | 60% | 60% | 0% | Tier 3 |
| Real Estate | 100% | 10% | 10% | 40% | 40% | 20% | 70% | 30% | Tier 3 |
| Retail | 100% | 10% | 0% | 70% | 20% | 10% | 90% | 50% | Tier 3 |
| Consumer Electronics | 80% | 30% | 30% | 50% | 30% | 50% | 80% | 10% | Tier 2 |
| Computer Hardware | 100% | 20% | 0% | 40% | 30% | 30% | 90% | 10% | Tier 2 |
| Automotive | 100% | 10% | 0% | 40% | 30% | 30% | 90% | 10% | Tier 2 |
| Furniture | 100% | 40% | 20% | 50% | 30% | 40% | 70% | 20% | Tier 2 |
| Hospitality | 90% | 30% | 0% | 60% | 20% | 40% | 70% | 20% | Tier 2 |
| Human Resources | 100% | 60% | 70% | 50% | 40% | 20% | 80% | 10% | Tier 2 |
| Beauty & Cosmetics | 90% | 30% | 20% | 30% | 20% | 40% | 90% | 10% | Tier 2 |
| Restaurants | 90% | 50% | 20% | 60% | 40% | 20% | 70% | 10% | Tier 2 |
| Apparel & Fashion | 90% | 10% | 20% | 40% | 20% | 30% | 80% | 10% | Tier 2 |
| Food & Beverages | 100% | 30% | 30% | 40% | 10% | 0% | 100% | 40% | Tier 2 |
| Design | 100% | 20% | 30% | 40% | 20% | 30% | 100% | 30% | Tier 2 |
| Education | 100% | 10% | 0% | 20% | 0% | 0% | 70% | 0% | Tier 2 |
Jewellery & Luxury and Real Estate share the top mean tier of 2.70, both with a median of 3. Retail (2.60, median 3) is close behind. A mid-table cluster from Consumer Electronics to Apparel & Fashion spans mean tiers of 2.20–2.50.
Nine Luxembourg sectors share the most accessible tier floor
| WAF | Antibot | CAPTCHA | JavaScript | Rate Limit | TLS | robots.txt | Blocks AI | Tier | |
|---|---|---|---|---|---|---|---|---|---|
| Equipment & Supplies | 90% | 40% | 20% | 30% | 10% | 0% | 80% | 0% | Tier 2 |
| Mail & Delivery | 70% | 40% | 10% | 10% | 0% | 20% | 80% | 0% | Tier 2 |
| Recruitment & Staffing | 100% | 40% | 60% | 30% | 0% | 10% | 90% | 20% | Tier 2 |
| Sporting Goods | 80% | 30% | 40% | 10% | 50% | 30% | 70% | 0% | Tier 2 |
| TV & Film | 100% | 50% | 10% | 50% | 30% | 20% | 60% | 10% | Tier 2 |
| Wholesalers | 80% | 10% | 20% | 30% | 10% | 30% | 70% | 10% | Tier 2 |
| Adult | 100% | 40% | 30% | 80% | 10% | 20% | 90% | 10% | Tier 2 |
| Construction | 90% | 10% | 30% | 50% | 10% | 0% | 80% | 10% | Tier 2 |
| Crafts | 100% | 30% | 20% | 60% | 30% | 10% | 80% | 20% | Tier 2 |
| Customer Services | 100% | 10% | 30% | 50% | 10% | 30% | 70% | 10% | Tier 2 |
| Import & Export | 100% | 30% | 30% | 40% | 0% | 40% | 80% | 10% | Tier 2 |
| Machinery | 90% | 0% | 10% | 50% | 0% | 0% | 90% | 10% | Tier 2 |
| Medical Devices | 100% | 0% | 10% | 60% | 10% | 0% | 90% | 10% | Tier 2 |
| Reference | 90% | 60% | 30% | 50% | 30% | 10% | 60% | 30% | Tier 2 |
| Telecom | 90% | 0% | 10% | 70% | 0% | 20% | 90% | 20% | Tier 2 |
Adult, Construction, Crafts, Customer Services, Import & Export, Machinery, Medical Devices, Reference, and Telecom all share the lowest mean tier of 1.90 among Luxembourg sectors in this group.