Overview
JavaScript and Rate Limiting both elevate Customer Services's active barrier profile
| Value | |
|---|---|
| WAF | 100% |
| Antibot | 22% |
| CAPTCHA | 22% |
| JavaScript | 55% |
| Rate Limiting | 38% |
| TLS | 17% |
WAF leads at 100%. JavaScript (55%) and Rate Limiting (38%) together form a multi-layer active stack — more than most sectors deploy alongside WAF.
Barrier Landscape
JavaScript and Rate Limiting define Customer Services's layered active defences
| Value | |
|---|---|
| WAF | 100% |
| Antibot | 22% |
| CAPTCHA | 22% |
| JavaScript | 55% |
| Rate Limiting | 38% |
| TLS | 17% |
| Has robots.txt | 76% |
| Blocks AI crawlers | 7% |
WAF leads at 100%. JavaScript (55%) and Rate Limiting (38%) are both well above the cross-sector norm, giving Customer Services a markedly heavier active barrier profile than most sectors.
Heavy barrier stacking is common in Customer Services — most sites carry three or more
| % of sites | |
|---|---|
| 0 barriers | 0% |
| 1 barrier | 25% |
| 2 barriers | 28% |
| 3 barriers | 25% |
| 4 barriers | 12% |
| 5 barriers | 10% |
| 6 barriers | 0% |
47% of Customer Services sites carry three or more barriers — well above the norm. Two-barrier stacks are most common. Just 0% of sites carry none.
Simple is the dominant access tier across Customer Services
| Mean recommended tier | |
|---|---|
| All sites | 1.67 |
A mean recommended tier of 1.67 and a median of Simple place Customer Services around the middle of the sectors surveyed. 54% of sites are Simple to access; 14% require Moderate or above.
Simple is the dominant tier in Customer Services, but Easy accounts for a significant share
| % of sites | |
|---|---|
| 1 - Simple | 54% |
| 2 - Easy | 32% |
| 3 - Moderate | 8% |
| 4 - Complex | 5% |
| 5 - Advanced | 1% |
Simple (54%) is the most common tier. Simple and Easy together account for 86% of sites, while 8% land at Moderate and 6% at Complex or Advanced.
robots.txt
Around three-quarters of Customer Services sites publish a robots.txt
| % of sites | |
|---|---|
| Publish a valid robots.txt | 76% |
76% of Customer Services sites have a valid robots.txt. The remaining 24% leave crawl policy entirely undefined — a meaningful gap for a sector of this scale.
Customer Services sites name a handful of agents in robots.txt — few commit to comprehensive policies
| % of sites | |
|---|---|
| 0 | 32% |
| 1 | 44% |
| 2–4 | 9% |
| 5–9 | 6% |
| 10–15 | 4% |
| 16+ | 5% |
The mean of 3.5 named agents reflects limited bot-policy investment in Customer Services. 32% of sites name no agents; 44% name exactly one. Only 5% maintain a list of 16 or more named bots.
SEO audit tools lead Customer Services's robots.txt mentions ahead of AI crawlers
| % of sites | |
|---|---|
| AI crawlers | 9% |
| SEO / search indexing | 9% |
| SEO audit tools | 15% |
| Commercial scrapers | 2% |
| Social / content scrapers | 1% |
| Archive / research crawlers | 3% |
| Monitoring / uptime bots | 1% |
SEO audit tools are mentioned by 15% of Customer Services sites — the leading bot category, ahead of AI crawlers at 9%. Figures are percentages of all 100 sites surveyed.
Uniform per-agent rules dominate Customer Services robots.txt files
| % of sites | |
|---|---|
| Uniform per-agent rules | 46% |
| Path-level mixed rules | 26% |
| Allow-only (no Disallow rules) | 2% |
| Mixed AI posture | 2% |
| Blanket Disallow: / | 0% |
Uniform per-agent rules lead at 46%, with path-level mixed rules at 26%. Mixed AI posture (2%) and blanket Disallow (0%) are minimal.
AhrefsBot leads named Disallow rules in Customer Services robots.txt
| Disallowed | Allowed (whitelisted) | |
|---|---|---|
| * | 65% | 28% |
| AhrefsBot | 9% | — |
| MJ12bot | 7% | — |
| BLEXBot | 6% | — |
| GPTBot | 4% | 2% |
| CCBot | 4% | 1% |
| ClaudeBot | 3% | 2% |
| PetalBot | 4% | — |
| Googlebot | 2% | 2% |
| facebookexternalhit | 1% | 3% |
| Amazonbot | 3% | 1% |
| SemrushBot | 4% | — |
| Google-Extended | 2% | 2% |
| dotbot | 3% | — |
| ChatGPT-User | 2% | 1% |
| Exabot | 3% | — |
| Bytespider | 3% | — |
| 008 | 2% | — |
| VelenPublicWebCrawler | 2% | — |
| DotBot | 2% | — |
| TurnitinBot | 2% | — |
| BDCbot | 2% | — |
| BLP_bbot | 2% | — |
| Barkrowler | 2% | — |
| GrapeshotCrawler | 2% | — |
Values are percentages of all 100 Customer Services sites surveyed. The wildcard (*) leads at 65% Disallow. Among named agents, AhrefsBot, MJ12bot, BLEXBot are the most frequently targeted — focused on non-AI crawlers.
Industry Cohort
Customer Services leads the Legal & Professional Services cohort on AI-blocking policy
| WAF | Antibot | CAPTCHA | JavaScript | Rate Limit | TLS | robots.txt | Blocks AI | Tier | |
|---|---|---|---|---|---|---|---|---|---|
| Consulting | 87% | 9% | 22% | 37% | 8% | 11% | 75% | 5% | Tier 1 |
| Customer Services | 100% | 22% | 22% | 55% | 38% | 17% | 76% | 7% | Tier 1 |
| Facilities Services | 94% | 9% | 23% | 27% | 12% | 10% | 75% | 1% | Tier 1 |
| Legal | 83% | 15% | 22% | 35% | 15% | 8% | 58% | 5% | Tier 2 |
Customer Services's 7% AI-blocking rate in robots.txt is the highest in the Legal & Professional Services cohort. WAF (100%) and JavaScript (55%) sit at the cohort on technical barriers.