Overview
CAPTCHA and JavaScript both elevate Fishery's active barrier profile
| Value | |
|---|---|
| WAF | 96% |
| Antibot | 13% |
| CAPTCHA | 37% |
| JavaScript | 32% |
| Rate Limiting | 18% |
| TLS | 9% |
WAF leads at 96%. CAPTCHA (37%) and JavaScript (32%) together form a multi-layer active stack — more than most sectors deploy alongside WAF.
Barrier Landscape
WAF leads Fishery at 96% — CAPTCHA is the main secondary layer
| Value | |
|---|---|
| WAF | 96% |
| Antibot | 13% |
| CAPTCHA | 37% |
| JavaScript | 32% |
| Rate Limiting | 18% |
| TLS | 9% |
| Has robots.txt | 69% |
| Blocks AI crawlers | 3% |
WAF dominates at 96%. CAPTCHA (37%) leads active secondary barriers, followed by JavaScript (32%). The remaining three barriers all sit at or below 18% — a concentrated rather than broadly stacked profile.
Fishery's barrier counts are spread — no single configuration dominates
| % of sites | |
|---|---|
| 0 barriers | 4% |
| 1 barrier | 34% |
| 2 barriers | 31% |
| 3 barriers | 20% |
| 4 barriers | 6% |
| 5 barriers | 5% |
| 6 barriers | 0% |
Barrier counts vary widely across Fishery sites. Single-barrier configurations are the modal count at 34%, but the distribution is broad. 31% of sites carry three or more.
Nearly all Fishery sites are Simple to access programmatically
| Mean recommended tier | |
|---|---|
| All sites | 1.03 |
A mean recommended tier of 1.03 and a median of Simple place Fishery among the most accessible sectors surveyed. 97% of sites are Simple to access; 0% require Moderate or above.
Simple tier covers more than three-quarters of Fishery sites
| % of sites | |
|---|---|
| 1 - Simple | 97% |
| 2 - Easy | 3% |
| 3 - Moderate | 0% |
| 4 - Complex | 0% |
| 5 - Advanced | 0% |
Simple (97%) is the most common tier. Simple and Easy together account for 100% of sites, while 0% land at Moderate and 0% at Complex or Advanced.
robots.txt
Around seven in ten of Fishery sites publish a robots.txt
| % of sites | |
|---|---|
| Publish a valid robots.txt | 69% |
69% of Fishery sites have a valid robots.txt. The remaining 31% leave crawl policy entirely undefined — a meaningful gap for a sector of this scale.
Most Fishery sites name one bot or none in their robots.txt
| % of sites | |
|---|---|
| 0 | 42% |
| 1 | 32% |
| 2–4 | 2% |
| 5–9 | 22% |
| 10–15 | 0% |
| 16+ | 2% |
42% of Fishery sites name zero agents and 32% name one. A mean of 2.0 is pulled up by a small number of sites with extensive policies, while the median stays at 1.
SEO audit tools lead Fishery's robots.txt mentions ahead of AI crawlers
| % of sites | |
|---|---|
| AI crawlers | 3% |
| SEO / search indexing | 5% |
| SEO audit tools | 19% |
| Commercial scrapers | 0% |
| Social / content scrapers | 0% |
| Archive / research crawlers | 1% |
| Monitoring / uptime bots | 0% |
SEO audit tools are mentioned by 19% of Fishery sites — the leading bot category, ahead of AI crawlers at 3%. Figures are percentages of all 100 sites surveyed.
Uniform per-agent rules dominate Fishery robots.txt files
| % of sites | |
|---|---|
| Uniform per-agent rules | 47% |
| Path-level mixed rules | 19% |
| Allow-only (no Disallow rules) | 3% |
| Mixed AI posture | 0% |
| Blanket Disallow: / | 0% |
Uniform per-agent rules lead at 47%, with path-level mixed rules at 19%. Mixed AI posture (0%) and blanket Disallow (0%) are minimal.
A cluster of four agents lead named Disallow rules in Fishery robots.txt
| Disallowed | Allowed (whitelisted) | |
|---|---|---|
| * | 54% | 22% |
| AhrefsBot | 16% | 1% |
| AhrefsSiteAudit | 16% | — |
| Nutch | 16% | — |
| adsbot-google | 16% | — |
| Googlebot | 2% | 1% |
| Amazonbot | 2% | — |
| Applebot-Extended | 2% | — |
| Bytespider | 2% | — |
| CCBot | 2% | — |
| ClaudeBot | 2% | — |
| GPTBot | 2% | — |
| Google-Extended | 2% | — |
| meta-externalagent | 2% | — |
| msnbot | 1% | 1% |
| Mediapartners-Google | 1% | 1% |
| Baiduspider | 2% | — |
| PetalBot | 2% | — |
| AdsBot-Google | 1% | 1% |
| Download Ninja | 1% | — |
| Fetch | 1% | — |
| HTTrack | 1% | — |
| MSIECrawler | 1% | — |
| Microsoft.URL.Control | 1% | — |
| Offline Explorer | 1% | — |
Values are percentages of all 100 Fishery sites surveyed. The wildcard (*) leads at 54% Disallow. Among named agents, AhrefsBot, AhrefsSiteAudit, Nutch and adsbot-google are the most frequently targeted, each at 16%.
Industry Cohort
Fishery is among the hardest sub-categories to access in Agriculture
| WAF | Antibot | CAPTCHA | JavaScript | Rate Limit | TLS | robots.txt | Blocks AI | Tier | |
|---|---|---|---|---|---|---|---|---|---|
| Agriculture | 97% | 12% | 30% | 41% | 20% | 12% | 90% | 12% | Tier 1 |
| Farms & Ranches | 96% | 12% | 32% | 26% | 24% | 14% | 67% | 6% | Tier 1 |
| Fishery | 96% | 13% | 37% | 32% | 18% | 9% | 69% | 3% | Tier 1 |
A median tier of Simple and mean of 1.03 make Fishery one of the harder sectors within Agriculture to access programmatically. JavaScript (32%) is elevated in the cohort.