Half of all sites layer two or more barriers — but nearly 1 in 5 remain completely open
| Value | |
|---|---|
| None | 18.5% of sites |
| 1 barrier | 30.7% of sites |
| 2 barriers | 25% of sites |
| 3 barriers | 12.1% of sites |
| 4 barriers | 5.4% of sites |
| 5 barriers | 7.4% of sites |
| 6 barriers | 0.8% of sites |
18.5% of landing pages deploy no detectable access barriers — a significant minority concentrated in static content, public sector, and information-first sectors where crawl access aligns with the site's distribution goals. The majority (81.5%) deploy at least one barrier, and more than half stack two or more. At the other extreme, 0.8% of sites run all six barriers simultaneously — an arms-race posture limited to the highest-value commercial categories.
Barrier profile
WAF is the only barrier that regularly operates alone — all others almost always need a partner
| Value | |
|---|---|
| WAF | Used alone: 33.4% |
| TLS | Used alone: 7.9% |
| JavaScript | Used alone: 0.6% |
| Rate Limiting | Used alone: 0.2% |
| CAPTCHA | Used alone: 0.1% |
| Antibot | Used alone: 0% |
A third of WAF-only sites deploy no other mechanism — WAF is widely treated as a sufficient standalone defence. Every other barrier tells the opposite story: TLS fingerprinting alone appears on 8% of TLS sites, and JavaScript, Rate Limiting, CAPTCHA, and Antibot practically never operate without WAF as a foundation. These barriers are architectural — they exist to compound, not to stand alone.
Top combinations
WAF + JavaScript covers one in five sites — the dominant two-layer stack
| WAF | Antibot | CAPTCHA | JavaScript | Rate Limiting | TLS | (none) | |
|---|---|---|---|---|---|---|---|
| WAF only | 30.8% | 0% | 0% | 0% | 0% | 0% | 0% |
| WAF + JavaScript | 10.2% | 0% | 0% | 10.2% | 0% | 0% | 0% |
| (none) | 0% | 0% | 0% | 0% | 0% | 0% | 18.5% |
| WAF + CAPTCHA | 3.25% | 0% | 3.25% | 0% | 0% | 0% | 0% |
| WAF + CAPTCHA + JavaScript | 1.47% | 0% | 1.47% | 1.47% | 0% | 0% | 0% |
| WAF + JavaScript + Rate Limiting | 1.37% | 0% | 0% | 1.37% | 1.37% | 0% | 0% |
| WAF + Antibot + CAPTCHA + Rate Limiting + TLS | 0.66% | 0.66% | 0.66% | 0% | 0.66% | 0.66% | 0% |
| WAF + Antibot + CAPTCHA + JavaScript + Rate Limiting | 0.66% | 0.66% | 0.66% | 0.66% | 0.66% | 0% | 0% |
| WAF + Rate Limiting + TLS | 1% | 0% | 0% | 0% | 1% | 1% | 0% |
| WAF + Antibot | 1.3% | 1.3% | 0% | 0% | 0% | 0% | 0% |
The three most common outcomes — WAF only, WAF + JavaScript, and no barriers at all — together account for nearly 70% of all landing pages. WAF-only (31%) and WAF+JavaScript (20%) reflect the dominant active-defence postures; (none) at 18.5% reflects the substantial portion of the web that remains open. The full-stack combination — WAF, Antibot, CAPTCHA, Rate Limiting, and TLS simultaneously — appears on 3.3% of sites, almost always in high-value commercial categories. The practical implication: a scraping pipeline that handles WAF and JavaScript rendering covers the majority of the defended web; anything beyond that requires a case-by-case approach.