All Industries — Overview
Where data is richest, defences run deepest
| WAF | Antibot | CAPTCHA | JavaScript | Rate Limit | TLS | robots.txt | Blocks AI | Tier | |
|---|---|---|---|---|---|---|---|---|---|
| Media & Publishing | 97% | 14% | 24% | 36% | 16% | 11% | 86% | 30% | Tier 2 |
| Retail & E-commerce | 93% | 23% | 27% | 44% | 31% | 19% | 77% | 12% | Tier 2 |
| Technology | 93% | 14% | 21% | 39% | 18% | 12% | 77% | 12% | Tier 1 |
| Financial Services | 90% | 21% | 14% | 37% | 16% | 12% | 74% | 5% | Tier 2 |
| Healthcare | 94% | 19% | 25% | 44% | 26% | 14% | 80% | 11% | Tier 1 |
| Manufacturing & Logistics | 90% | 16% | 22% | 38% | 18% | 14% | 71% | 6% | Tier 1 |
| Travel & Hospitality | 85% | 23% | 16% | 50% | 29% | 18% | 68% | 9% | Tier 2 |
| Entertainment | 96% | 27% | 24% | 49% | 30% | 14% | 75% | 16% | Tier 1 |
| Education | 92% | 17% | 19% | 38% | 17% | 10% | 76% | 13% | Tier 1 |
| Design & Creative Arts | 97% | 18% | 29% | 38% | 20% | 12% | 80% | 11% | Tier 1 |
| Public Sector | 89% | 11% | 16% | 35% | 16% | 12% | 73% | 7% | Tier 1 |
| Legal & Professional Svcs | 91% | 14% | 22% | 39% | 18% | 12% | 71% | 5% | Tier 1 |
| Non-profit & Civil Society | 95% | 22% | 27% | 40% | 21% | 14% | 82% | 10% | Tier 1 |
| Agriculture | 96% | 12% | 33% | 33% | 21% | 12% | 75% | 7% | Tier 1 |
| Energy & Utilities | 86% | 16% | 15% | 38% | 16% | 11% | 63% | 4% | Tier 1 |
| Construction & Environment | 92% | 17% | 19% | 47% | 23% | 11% | 76% | 9% | Tier 2 |
| Recruitment & HR | 92% | 19% | 20% | 35% | 18% | 15% | 75% | 6% | Tier 1 |
| Automotive | 92% | 16% | 9% | 44% | 38% | 16% | 80% | 22% | Tier 2 |
| Real Estate | 92% | 24% | 19% | 48% | 39% | 15% | 75% | 17% | Tier 2 |
| Telecom | 95% | 19% | 18% | 45% | 16% | 11% | 79% | 6% | Tier 2 |
WAF adoption is near-universal across all 20 sectors — the real variation is in what comes next. Entertainment, retail, and travel deploy meaningful antibot and rate limiting; public sector, energy, and agriculture rely almost entirely on CDN-bundled WAF coverage. AI blocking is nascent everywhere, with media and entertainment the outliers. Each row is the sector average across all sub-categories; expand any section below for sub-category detail.
Media & Publishing
Media sites police AI crawlers with robots.txt, not middleware
| WAF | Antibot | CAPTCHA | JavaScript | Rate Limit | TLS | robots.txt | Blocks AI | Tier | |
|---|---|---|---|---|---|---|---|---|---|
| Advertising & Marketing | 97% | 23% | 32% | 31% | 21% | 14% | 87% | 6% | Tier 1 |
| Mass Media | 98% | 12% | 21% | 49% | 16% | 8% | 92% | 43% | Tier 2 |
| Newspapers | 96% | 8% | 21% | 36% | 15% | 10% | 90% | 64% | Tier 2 |
| Public Relations | 95% | 10% | 14% | 25% | 14% | 14% | 76% | 6% | Tier 1 |
| Publishing | 95% | 16% | 24% | 45% | 21% | 11% | 92% | 49% | Tier 2 |
| Writing & Editing | 98% | 14% | 33% | 31% | 11% | 6% | 79% | 11% | Tier 1 |
WAF coverage is near-total across every Media & Publishing sub-category (95–98%), yet antibot deployment sits at just 8–23% — the lowest of any major sector. The real defence story here is robots.txt: 64% of newspaper sites and 49% of publishers explicitly block at least one AI crawler by name, reflecting a deliberate choice to use policy rather than middleware to resist AI training scrapes.
Retail & E-commerce
Retail relies on rate limiting and JavaScript over dedicated antibot
| WAF | Antibot | CAPTCHA | JavaScript | Rate Limit | TLS | robots.txt | Blocks AI | Tier | |
|---|---|---|---|---|---|---|---|---|---|
| Apparel & Fashion | 91% | 21% | 15% | 49% | 54% | 32% | 69% | 4% | Tier 3 |
| Beauty & Cosmetics | 91% | 30% | 36% | 40% | 31% | 30% | 76% | 13% | Tier 2 |
| Computer & Video Games | 94% | 27% | 26% | 44% | 22% | 13% | 81% | 23% | Tier 1 |
| Computer Hardware | 93% | 30% | 20% | 42% | 33% | 15% | 73% | 13% | Tier 2 |
| Computers & Electronics | 86% | 13% | 17% | 34% | 18% | 15% | 73% | 9% | Tier 2 |
| Consumer Electronics | 92% | 27% | 27% | 46% | 29% | 20% | 83% | 9% | Tier 2 |
| Equipment & Supplies | 96% | 27% | 31% | 41% | 26% | 14% | 68% | 8% | Tier 2 |
| Food & Beverages | 92% | 24% | 26% | 51% | 31% | 11% | 68% | 23% | Tier 2 |
| Furniture | 97% | 33% | 28% | 49% | 40% | 21% | 67% | 5% | Tier 2 |
| Jewelry & Luxury | 83% | 28% | 23% | 41% | 42% | 34% | 80% | 9% | Tier 2 |
| Packaging & Containers | 95% | 18% | 35% | 42% | 17% | 11% | 80% | 6% | Tier 1 |
| Retail | 96% | 19% | 15% | 51% | 33% | 20% | 85% | 29% | Tier 2 |
| Sporting Goods | 89% | 26% | 24% | 41% | 49% | 32% | 78% | 7% | Tier 2 |
| Sports | 98% | 11% | 15% | 61% | 23% | 11% | 89% | 33% | Tier 1 |
| Textiles & Nonwovens | 92% | 6% | 38% | 35% | 17% | 14% | 78% | 7% | Tier 1 |
| Tobacco | 98% | 30% | 52% | 38% | 30% | 15% | 76% | 3% | Tier 1 |
| Wholesalers | 94% | 24% | 22% | 44% | 30% | 13% | 80% | 7% | Tier 2 |
| Wine & Spirits | 95% | 24% | 28% | 47% | 32% | 15% | 84% | 14% | Tier 1 |
Rate limiting (31%) and JavaScript challenges (44%) are retail's de-facto defences — blunter and cheaper to deploy than dedicated antibot middleware. Sports sites lead AI blocking at 33%, hinting at real-time odds and fantasy sports data concerns, while furniture (33%) and beauty-and-cosmetics (28%) carry the highest antibot rates, both driven by high-margin price-scraping risk.
Technology
Online services block AI crawlers at twice the tech-sector average
| WAF | Antibot | CAPTCHA | JavaScript | Rate Limit | TLS | robots.txt | Blocks AI | Tier | |
|---|---|---|---|---|---|---|---|---|---|
| Biotech | 89% | 13% | 19% | 39% | 17% | 13% | 76% | 14% | Tier 1 |
| Software & Dev | 95% | 17% | 19% | 30% | 18% | 13% | 83% | 14% | Tier 2 |
| Information Technology | 94% | 13% | 15% | 36% | 21% | 14% | 68% | 7% | Tier 2 |
| Market Research | 94% | 16% | 22% | 44% | 25% | 14% | 71% | 0% | Tier 1 |
| Nanotechnology | 95% | 8% | 33% | 31% | 10% | 9% | 79% | 6% | Tier 1 |
| Online Services | 93% | 9% | 16% | 50% | 19% | 12% | 86% | 36% | Tier 2 |
| Security Products | 91% | 19% | 26% | 40% | 15% | 12% | 73% | 4% | Tier 1 |
Online services — primarily content and SaaS platforms — show 36% AI blocking, far above the 12% sector average. Security product vendors (19% antibot) are notably under-protected relative to their security-first brand positioning. Biotech and nanotechnology sites sit at Tier 1 complexity: academic-adjacent content with minimal commercial scraping risk.
Financial Services
Finance's WAF compliance masks chronic antibot underinvestment
| WAF | Antibot | CAPTCHA | JavaScript | Rate Limit | TLS | robots.txt | Blocks AI | Tier | |
|---|---|---|---|---|---|---|---|---|---|
| Accounting & Auditing | 93% | 15% | 13% | 40% | 8% | 10% | 63% | 1% | Tier 1 |
| Banking | 86% | 19% | 10% | 36% | 12% | 11% | 72% | 4% | Tier 2 |
| Finance | 88% | 27% | 15% | 36% | 22% | 16% | 82% | 13% | Tier 2 |
| Insurance | 86% | 26% | 19% | 37% | 17% | 13% | 71% | 3% | Tier 2 |
| Investment | 94% | 23% | 14% | 41% | 19% | 9% | 76% | 5% | Tier 1 |
| Venture Capital | 92% | 15% | 14% | 33% | 20% | 15% | 79% | 2% | Tier 1 |
WAF coverage (90%) reflects mandatory compliance posture, but antibot middleware (21%) is a fraction of what financial data risk would suggest. Finance and investment sub-categories lead the sector in antibot (27%) — driven by price-data and prospectus-scraping risk — while accounting and insurance lag below 15%. AI blocking is negligible (5% average): financial institutions are not yet treating AI training scrapes as a threat worth blocking.
Healthcare
Consumer-facing healthcare leads the sector in antibot; clinical and pharma trail
| WAF | Antibot | CAPTCHA | JavaScript | Rate Limit | TLS | robots.txt | Blocks AI | Tier | |
|---|---|---|---|---|---|---|---|---|---|
| Healthcare | 93% | 17% | 21% | 35% | 27% | 21% | 78% | 14% | Tier 2 |
| Medical Devices | 92% | 13% | 22% | 45% | 16% | 8% | 81% | 8% | Tier 1 |
| Mental Health | 95% | 16% | 24% | 40% | 18% | 6% | 85% | 12% | Tier 1 |
| Pharmaceuticals | 84% | 20% | 25% | 54% | 27% | 8% | 62% | 5% | Tier 1 |
| Veterinary | 100% | 24% | 29% | 43% | 35% | 25% | 89% | 5% | Tier 1 |
| Wellness | 98% | 23% | 31% | 48% | 34% | 16% | 83% | 22% | Tier 2 |
WAF coverage (94%) reflects broad regulatory compliance, but antibot is unevenly distributed: clinical healthcare and mental health sit at 18–19%, while wellness (24%) and veterinary (24%) — consumer-facing sub-categories with review, pricing, and appointment scraping risk — lead the sector. AI blocking averages 11%, with wellness at 22%, suggesting consumer-facing health content is beginning to attract AI training attention.
Manufacturing & Logistics
Industrial sites run minimal access controls — product catalogues are rarely scraping targets
| WAF | Antibot | CAPTCHA | JavaScript | Rate Limit | TLS | robots.txt | Blocks AI | Tier | |
|---|---|---|---|---|---|---|---|---|---|
| Aerospace & Defense | 87% | 17% | 26% | 36% | 21% | 15% | 67% | 6% | Tier 1 |
| Chemicals | 83% | 8% | 19% | 31% | 7% | 8% | 71% | 8% | Tier 1 |
| Civil Engineering | 89% | 13% | 22% | 38% | 19% | 14% | 60% | 5% | Tier 1 |
| Import & Export | 93% | 19% | 17% | 35% | 17% | 13% | 69% | 6% | Tier 1 |
| Machinery | 91% | 13% | 17% | 49% | 20% | 15% | 80% | 10% | Tier 2 |
| Mail & Package Delivery | 91% | 15% | 19% | 37% | 17% | 15% | 70% | 3% | Tier 1 |
| Manufacturing | 92% | 20% | 19% | 45% | 24% | 18% | 75% | 9% | Tier 2 |
| Maritime | 87% | 22% | 24% | 41% | 19% | 12% | 62% | 8% | Tier 1 |
| Metals & Mining | 87% | 26% | 30% | 39% | 25% | 16% | 72% | 4% | Tier 1 |
| Plastics & Polymers | 98% | 10% | 29% | 34% | 13% | 10% | 80% | 4% | Tier 1 |
| Transportation & Logistics | 78% | 15% | 16% | 35% | 17% | 18% | 67% | 2% | Tier 1 |
| Warehousing | 98% | 18% | 27% | 38% | 22% | 8% | 75% | 4% | Tier 1 |
Metals and mining leads antibot at 26% — likely driven by commodity pricing data — but most sub-categories cluster at 12–18%. Mail and package delivery and manufacturing break Tier 2 complexity, hinting at dynamic shipment tracking and order management systems beneath thin marketing facades. AI blocking averages just 6%, the lowest of any large-category sector in the report.
Travel & Hospitality
Travel booking engines hide pricing behind JavaScript, not antibot middleware
| WAF | Antibot | CAPTCHA | JavaScript | Rate Limit | TLS | robots.txt | Blocks AI | Tier | |
|---|---|---|---|---|---|---|---|---|---|
| Airlines | 76% | 24% | 11% | 51% | 27% | 22% | 69% | 9% | Tier 2 |
| Events Services | 83% | 17% | 21% | 41% | 26% | 12% | 65% | 3% | Tier 1 |
| Hospitality | 86% | 18% | 17% | 47% | 27% | 19% | 76% | 9% | Tier 2 |
| Recreational Facilities | 95% | 23% | 21% | 51% | 21% | 11% | 71% | 6% | Tier 1 |
| Restaurants | 91% | 28% | 19% | 57% | 41% | 15% | 50% | 5% | Tier 2 |
| Travel & Tourism | 81% | 27% | 8% | 55% | 31% | 26% | 78% | 21% | Tier 2 |
Travel and hospitality has the highest JavaScript challenge rate of any sector (50%) and the second-lowest WAF coverage (85%). Airlines, travel-and-tourism, and restaurants all run Tier 2 complexity — booking engines and reservation systems that require full JavaScript execution to scrape. Antibot (23%) is moderate; travel-and-tourism leads AI blocking at 21%, reflecting concern about pricing and review data entering AI training sets.
Entertainment
Gambling and adult content run the tightest access controls in entertainment
| WAF | Antibot | CAPTCHA | JavaScript | Rate Limit | TLS | robots.txt | Blocks AI | Tier | |
|---|---|---|---|---|---|---|---|---|---|
| Adult | 98% | 36% | 33% | 36% | 34% | 20% | 85% | 9% | Tier 1 |
| Entertainment | 92% | 17% | 18% | 46% | 23% | 13% | 77% | 30% | Tier 2 |
| Gambling | 95% | 31% | 24% | 53% | 35% | 14% | 57% | 6% | Tier 1 |
| Music | 96% | 21% | 19% | 52% | 26% | 12% | 84% | 25% | Tier 2 |
| TV & Movies | 99% | 30% | 25% | 59% | 33% | 13% | 70% | 11% | Tier 1 |
Entertainment polarises sharply: adult (36% antibot) and gambling (31%) run the sector's highest middleware investment, driven by payment-gate protection and age-verification compliance. Music and TV/Movies cluster around 23–25%, where antibot primarily guards subscription content. General entertainment sites block AI crawlers most aggressively at 30%, suggesting that broad content libraries are a primary AI training-data target.
Education
Distance learning blocks AI crawlers at twice education's sector average
| WAF | Antibot | CAPTCHA | JavaScript | Rate Limit | TLS | robots.txt | Blocks AI | Tier | |
|---|---|---|---|---|---|---|---|---|---|
| Distance Learning | 97% | 27% | 37% | 43% | 26% | 16% | 85% | 25% | Tier 2 |
| Education | 98% | 17% | 18% | 37% | 18% | 10% | 81% | 15% | Tier 2 |
| Foreign Language | 96% | 15% | 20% | 42% | 14% | 13% | 82% | 6% | Tier 1 |
| Libraries | 92% | 16% | 9% | 36% | 16% | 11% | 69% | 8% | Tier 1 |
| Reference | 81% | 15% | 17% | 31% | 16% | 3% | 60% | 17% | Tier 1 |
| Science | 90% | 15% | 11% | 35% | 15% | 9% | 79% | 10% | Tier 2 |
| Training & Certification | 93% | 15% | 19% | 41% | 17% | 11% | 75% | 7% | Tier 1 |
Distance learning (27% antibot, 25% AI blocking) stands apart from an otherwise low-security sector — online course platforms face real competitive scraping risk for curriculum content and pricing data. Libraries (15% antibot, 8% AI blocking) and reference sites sit at Tier 1 complexity with minimal barriers, reflecting their open-access academic mission and a culture of free information exchange.
Design & Creative Arts
Creative agencies inherit WAF coverage from their CMS — antibot is nearly absent
| WAF | Antibot | CAPTCHA | JavaScript | Rate Limit | TLS | robots.txt | Blocks AI | Tier | |
|---|---|---|---|---|---|---|---|---|---|
| Architecture | 95% | 21% | 33% | 40% | 20% | 10% | 84% | 12% | Tier 1 |
| Ceramics & Pottery | 97% | 16% | 38% | 47% | 17% | 12% | 77% | 5% | Tier 1 |
| Comics & Animation | 93% | 15% | 28% | 34% | 10% | 7% | 77% | 6% | Tier 1 |
| Crafts | 98% | 21% | 34% | 44% | 24% | 12% | 82% | 18% | Tier 1 |
| Design | 99% | 24% | 27% | 40% | 21% | 11% | 89% | 14% | Tier 1 |
| Graphic Design | 99% | 11% | 21% | 19% | 12% | 9% | 87% | 9% | Tier 1 |
| Performing Arts | 93% | 11% | 19% | 39% | 19% | 11% | 75% | 7% | Tier 1 |
| Photography | 95% | 19% | 28% | 42% | 42% | 32% | 68% | 15% | Tier 1 |
| Printing | 98% | 19% | 28% | 34% | 13% | 10% | 81% | 6% | Tier 1 |
| Visual Art | 98% | 20% | 31% | 43% | 20% | 10% | 84% | 22% | Tier 1 |
WAF coverage is near-universal (97%) but almost certainly bundled with hosted CMS or CDN services — antibot (18%) sits at one of the lowest rates across all sectors. Visual art sites lead AI blocking at 22%, reflecting concern about high-resolution image scraping for AI training sets. All 10 sub-categories are Tier 1, the lowest complexity profile of any industry group in the report.
Public Sector
Government sites have the lowest antibot deployment of any sector
| WAF | Antibot | CAPTCHA | JavaScript | Rate Limit | TLS | robots.txt | Blocks AI | Tier | |
|---|---|---|---|---|---|---|---|---|---|
| Government | 80% | 10% | 8% | 33% | 15% | 12% | 64% | 10% | Tier 2 |
| Military | 77% | 11% | 16% | 21% | 19% | 17% | 62% | 7% | Tier 1 |
| Political Orgs | 97% | 13% | 23% | 47% | 16% | 9% | 79% | 6% | Tier 1 |
| Public Policy | 96% | 9% | 9% | 28% | 11% | 9% | 86% | 8% | Tier 1 |
| Public Safety | 94% | 12% | 25% | 44% | 18% | 14% | 74% | 6% | Tier 1 |
Public sector antibot (11%) is the lowest median of any industry group in this report, below even agriculture. WAF coverage (89%) is largely compliance-driven rather than threat-responsive. Military sites run moderately higher antibot (17%) but still trail private-sector equivalents significantly. Political organisations show low AI blocking (10%), leaving campaign content freely accessible to training scrapes despite clear sensitivity around electoral information.
Legal & Professional Services
Professional services invest less in web security than their data sensitivity warrants
| WAF | Antibot | CAPTCHA | JavaScript | Rate Limit | TLS | robots.txt | Blocks AI | Tier | |
|---|---|---|---|---|---|---|---|---|---|
| Consulting | 87% | 9% | 22% | 37% | 8% | 11% | 75% | 5% | Tier 1 |
| Customer Services | 100% | 22% | 22% | 55% | 38% | 17% | 76% | 7% | Tier 1 |
| Facilities Services | 94% | 9% | 23% | 27% | 12% | 10% | 75% | 1% | Tier 1 |
| Legal | 83% | 15% | 22% | 35% | 15% | 8% | 58% | 5% | Tier 2 |
Legal and consulting sites cluster at 14–22% antibot — below financial services, despite often hosting equally sensitive commercial intelligence: case databases, market reports, and contract templates. Customer services platforms lead the sector at 22% antibot and 7% AI blocking, protecting interaction data and knowledge bases used by competing service providers.
Non-profit & Civil Society
Museums lead non-profit AI blocking; charities barely invest
| WAF | Antibot | CAPTCHA | JavaScript | Rate Limit | TLS | robots.txt | Blocks AI | Tier | |
|---|---|---|---|---|---|---|---|---|---|
| Museums | 95% | 27% | 32% | 43% | 22% | 13% | 78% | 13% | Tier 1 |
| Nonprofit Orgs | 96% | 18% | 19% | 45% | 24% | 17% | 83% | 10% | Tier 2 |
| Religion & Belief | 96% | 22% | 26% | 32% | 19% | 13% | 89% | 10% | Tier 1 |
| Social & Charitable | 94% | 22% | 31% | 38% | 20% | 11% | 79% | 5% | Tier 1 |
Museums run 27% antibot and 13% AI blocking — the highest figures in the sector — driven by concern for digital collections and high-resolution image assets. Social and charitable organisations sit below the sector average on every metric, consistent with limited IT budgets and minimal commercial scraping incentive. Overall, non-profits rely almost entirely on WAFs while leaving more active defences largely undeployed.
Agriculture
Agriculture's web security posture matches its scraping risk: near zero
| WAF | Antibot | CAPTCHA | JavaScript | Rate Limit | TLS | robots.txt | Blocks AI | Tier | |
|---|---|---|---|---|---|---|---|---|---|
| Agriculture | 97% | 12% | 30% | 41% | 20% | 12% | 90% | 12% | Tier 1 |
| Farms & Ranches | 96% | 12% | 32% | 26% | 24% | 14% | 67% | 6% | Tier 1 |
| Fishery | 96% | 13% | 37% | 32% | 18% | 9% | 69% | 3% | Tier 1 |
Agriculture clusters at 12–13% antibot and near-zero AI blocking (7%), all at Tier 1 complexity. High robots.txt coverage (75%) suggests these are mostly standard CMS deployments following default configuration rather than deliberate policy. There is no meaningful commercial incentive to scrape agricultural content at scale, and the sector's web security posture reflects that reality.
Energy & Utilities
Energy and utilities carries the weakest web security posture of any major sector
| WAF | Antibot | CAPTCHA | JavaScript | Rate Limit | TLS | robots.txt | Blocks AI | Tier | |
|---|---|---|---|---|---|---|---|---|---|
| Oil & Gas | 86% | 15% | 17% | 38% | 15% | 12% | 59% | 4% | Tier 1 |
| Public Utility | 83% | 19% | 13% | 37% | 17% | 10% | 53% | 3% | Tier 1 |
| Renewable Energy | 89% | 14% | 16% | 38% | 15% | 11% | 78% | 5% | Tier 1 |
Energy and utilities has the lowest WAF coverage (86%) of any multi-category sector group, and antibot (16%) sits near the bottom across all industries. Oil and gas robots.txt coverage drops to 59%, a signal of older web infrastructure not following modern CMS defaults. AI blocking is negligible at 4% — the lowest average of any sector in the report.
Construction & Environment
Construction's B2B content model leaves little incentive for antibot investment
| WAF | Antibot | CAPTCHA | JavaScript | Rate Limit | TLS | robots.txt | Blocks AI | Tier | |
|---|---|---|---|---|---|---|---|---|---|
| Construction & Maintenance | 94% | 21% | 24% | 52% | 29% | 9% | 75% | 7% | Tier 2 |
| Ecology & Environment | 90% | 13% | 13% | 42% | 16% | 13% | 76% | 10% | Tier 1 |
Both sub-categories show 92% WAF and around 47% JavaScript challenges, consistent with modern CMS hosting defaults. Antibot is low (17–21%) — construction and maintenance information carries minimal commercial scraping value. Ecology and environment sites outpace construction slightly on robots.txt (82% vs 70%), reflecting more content-oriented publishing practices.
Recruitment & HR
Job boards deploy antibot at average rates despite being prime scraping targets
| WAF | Antibot | CAPTCHA | JavaScript | Rate Limit | TLS | robots.txt | Blocks AI | Tier | |
|---|---|---|---|---|---|---|---|---|---|
| Human Resources | 93% | 24% | 25% | 38% | 20% | 14% | 70% | 2% | Tier 1 |
| Outsourcing | 96% | 11% | 18% | 28% | 15% | 12% | 79% | 2% | Tier 1 |
| Recruitment & Staffing | 88% | 21% | 18% | 39% | 18% | 18% | 76% | 15% | Tier 1 |
Despite job listings being among the most scraped content on the web, recruitment and HR antibot (19%) sits at the industry median. Rate limiting (18%) and JavaScript (35%) are the primary defences against bulk job-data harvesting. Recruitment and staffing sites lead AI blocking at 15%, reflecting growing concern about CV data and candidate profiles entering AI training sets.
Automotive
Automotive defends with rate limiting and JavaScript — antibot lags the inventory scraping risk
| WAF | Antibot | CAPTCHA | JavaScript | Rate Limit | TLS | robots.txt | Blocks AI | Tier | |
|---|---|---|---|---|---|---|---|---|---|
| Automotive | 92% | 16% | 9% | 44% | 38% | 16% | 80% | 22% | Tier 2 |
The automotive sector shows 92% WAF, 44% JavaScript challenges, and 38% rate limiting — reflecting the dynamic pricing and vehicle configuration tools that populate dealer and marketplace sites. Antibot (16%) is below average given the commercial scraping risk of inventory and pricing data. AI blocking at 22% is above the cross-industry median, suggesting automotive content is already a recognised AI training-data target.
Real Estate
Listing data is too valuable to leave unguarded — but real estate's antibot still lags
| WAF | Antibot | CAPTCHA | JavaScript | Rate Limit | TLS | robots.txt | Blocks AI | Tier | |
|---|---|---|---|---|---|---|---|---|---|
| Real Estate | 92% | 24% | 19% | 48% | 39% | 15% | 75% | 17% | Tier 2 |
Real estate runs 39% rate limiting and 48% JavaScript challenges — both among the highest across all sectors — consistent with the scale of automated listing scraping by property aggregators and market intelligence tools. Antibot (24%) is moderate; the Tier 2 median reflects dynamic listing systems that require significant execution overhead to crawl. AI blocking at 17% is above average, with property descriptions and pricing data increasingly drawn into training sets.
Telecom
Telecoms cover WAF universally but leave pricing data exposed to scraping
| WAF | Antibot | CAPTCHA | JavaScript | Rate Limit | TLS | robots.txt | Blocks AI | Tier | |
|---|---|---|---|---|---|---|---|---|---|
| Telecom | 95% | 19% | 18% | 45% | 16% | 11% | 79% | 6% | Tier 2 |
Telecom shows 95% WAF and 45% JavaScript challenges, but antibot sits at 19% — modest for an industry where plan pricing, coverage maps, and device offers are routinely harvested by comparison engines and competitor intelligence tools. AI blocking (6%) is low, suggesting telecoms are not yet treating AI training-data scraping as a meaningful threat worth blocking at the robots.txt level.