Overview
JavaScript is Japan's most common secondary barrier
| Value | |
|---|---|
| WAF | 86.4% |
| Antibot | 15.8% |
| CAPTCHA | 18.7% |
| JavaScript | 37.4% |
| Rate Limiting | 10.3% |
| TLS | 15.4% |
WAF adoption stands at 86.4% of Japan's 1,111 surveyed sites. JavaScript leads secondary barriers at 37.4%, followed by CAPTCHA at 18.7% and Antibot at 15.8%.
Barrier Landscape
JavaScript tops Japan's active secondary defences
| Value | |
|---|---|
| WAF | 86.4% |
| Antibot | 15.8% |
| CAPTCHA | 18.7% |
| JavaScript | 37.4% |
| Rate Limiting | 10.3% |
| TLS | 15.4% |
| Has robots.txt | 73.5% |
| Blocks AI crawlers | 12.3% |
WAF at 86.4% is the dominant barrier across Japan's surveyed sites. JavaScript (37.4%) is the most-deployed secondary barrier; CAPTCHA (18.7%) and Antibot (15.8%) follow.
Two-barrier sites are the most common configuration
| % of sites | |
|---|---|
| 0 barriers | 11.1% |
| 1 barrier | 32% |
| 2 barriers | 32.7% |
| 3 barriers | 14.3% |
| 4 barriers | 6.8% |
| 5 barriers | 2.8% |
| 6 barriers | 0.5% |
11.1% of Japan sites present no detectable barriers. The modal outcome is two barriers, covering 32.7% of sites. Sites with three or more barriers account for 24.4%.
Most Japan sites are Easy to access
| Mean recommended tier | |
|---|---|
| All sites | 1.6 |
A mean of 1.58 and median of 1 characterise Japan's access difficulty. Tier 1 accounts for 55.2% of sites; the 11.4% at Tier 3–5 pull the mean upward.
Nearly nine in ten sites are Simple or Easy to access
| % of sites | |
|---|---|
| 1 - Simple | 55.2% |
| 2 - Easy | 33.5% |
| 3 - Moderate | 9.3% |
| 4 - Complex | 1.8% |
| 5 - Advanced | 0.3% |
Tier 1 covers 55.2% of Japan's sites; Tier 2 adds 33.5%, bringing the combined Tier 1+2 share to 88.7%. Tier 3 accounts for 9.3%, Tier 4 for 1.8%, Tier 5 for 0.3%.
robots.txt
AI crawlers are the most-cited bot category in robots.txt
| % of sites | |
|---|---|
| AI crawlers | 13.8% |
| SEO / search indexing | 10.9% |
| SEO audit tools | 12.1% |
| Commercial scrapers | 6.9% |
| Social / content scrapers | 5.7% |
| Archive / research crawlers | 4.1% |
| Monitoring / uptime bots | 0.5% |
AI crawlers are mentioned by 13.8% of Japan's 1,111 surveyed sites — the most of any bot category. SEO / search indexing follow at 10.9%. Figures are percentages of all surveyed sites, including those without robots.txt.
Uniform rules are the most common robots.txt posture
| % of sites | |
|---|---|
| Uniform per-agent rules (no path mixing) | 38.3% |
| Path-level mixed rules for same agent | 30.2% |
| Allow-only (no disallow rules) | 2.1% |
| Mixed AI posture (block some, allow others) | 2% |
| Blanket Disallow: / | 2.8% |
26.5% of Japan's sites have no robots.txt. Uniform rules is the most common posture at 38.3% of all sites, followed by Path-mixed rules at 30.2%. Blanket Disallow: / covers 2.8%.
GPTBot faces the most Disallow rules
| Disallowed | Allowed (whitelisted) | |
|---|---|---|
| GPTBot | 10.3% | 1.5% |
| CCBot | 7.4% | 0.6% |
| ClaudeBot | 7.4% | 0.7% |
| Google-Extended | 7.7% | 0.7% |
| Bytespider | 5.9% | 0.3% |
| meta-externalagent | 2.8% | 0.2% |
| ChatGPT-User | 5.2% | 2.2% |
| PerplexityBot | 5.1% | 1.7% |
| Applebot-Extended | 4.4% | 0.2% |
| OAI-SearchBot | 3.2% | 1.7% |
| anthropic-ai | 3.9% | 0.5% |
Of Japan's 1,111 surveyed sites, GPTBot is Disallowed by 10.3% and explicitly Allowed by 1.5%. ChatGPT-User has the highest explicit Allow rate at 2.2%. All values include sites without robots.txt.
Four sectors name bots on every site
| % of sites | |
|---|---|
| Computer & Video Games | 100% |
| Design | 100% |
| Newspapers | 100% |
| Retail | 100% |
| Entertainment | 90% |
| General | 90% |
| Healthcare | 90% |
| Mass Media | 90% |
| Online Services | 90% |
| Publishing | 90% |
| Telecom | 90% |
| Agriculture | 80% |
| Architecture | 80% |
| Automotive | 80% |
| Beauty & Cosmetics | 80% |
Four sectors each show a 100% named-bot Disallow rate — every site in those sectors names at least one bot to restrict in robots.txt. The next sectors in the ranking stand at 90%.
Architecture leads explicit bot Allow rules
| % of sites | |
|---|---|
| Architecture | 70% |
| Automotive | 70% |
| Photography | 70% |
| Publishing | 70% |
| Retail | 70% |
| Travel & Tourism | 70% |
| Airlines | 60% |
| Farms & Ranches | 60% |
| Investment | 60% |
| Renewable Energy | 60% |
| Science | 60% |
| Software & Dev | 60% |
| Sports | 60% |
| Telecom | 60% |
| Veterinary | 60% |
Architecture has the highest named-bot Allow rate in Japan at 70% of sites in that sector. Automotive follows at 70%.
Industry Patterns
Jewellery & Luxury is the hardest sector to access programmatically
| WAF | Antibot | CAPTCHA | JavaScript | Rate Limit | TLS | robots.txt | Blocks AI | Tier | |
|---|---|---|---|---|---|---|---|---|---|
| Jewellery & Luxury | 100% | 20% | 10% | 30% | 30% | 40% | 60% | 0% | Tier 3 |
| Real Estate | 90% | 40% | 40% | 70% | 40% | 10% | 70% | 20% | Tier 3 |
| Human Resources | 100% | 70% | 70% | 40% | 60% | 30% | 70% | 10% | Tier 3 |
| Apparel & Fashion | 100% | 20% | 20% | 60% | 40% | 50% | 70% | 10% | Tier 2 |
| Hospitality | 80% | 30% | 0% | 50% | 20% | 40% | 60% | 10% | Tier 2 |
| Computer Hardware | 80% | 10% | 0% | 20% | 10% | 30% | 80% | 10% | Tier 2 |
| Beauty & Cosmetics | 80% | 40% | 30% | 50% | 10% | 20% | 80% | 20% | Tier 2 |
| Consumer Electronics | 90% | 10% | 20% | 50% | 10% | 20% | 80% | 10% | Tier 2 |
| Furniture | 100% | 10% | 10% | 60% | 20% | 20% | 60% | 20% | Tier 2 |
| Automotive | 90% | 0% | 0% | 30% | 30% | 10% | 90% | 10% | Tier 2 |
| Design | 100% | 20% | 30% | 40% | 20% | 30% | 100% | 30% | Tier 2 |
| Equipment & Supplies | 80% | 0% | 0% | 30% | 20% | 0% | 60% | 20% | Tier 2 |
| Mail & Delivery | 70% | 40% | 10% | 10% | 0% | 20% | 80% | 0% | Tier 2 |
| Maritime | 80% | 10% | 0% | 30% | 0% | 30% | 80% | 10% | Tier 2 |
| Restaurants | 70% | 10% | 10% | 30% | 10% | 20% | 70% | 10% | Tier 2 |
Jewellery & Luxury averages mean tier 2.90 across Japan's surveyed sites — the highest of any industry. Real Estate follows at 2.70. Industries ranked by mean recommended browser tier.
Biotech is the most accessible sector
| WAF | Antibot | CAPTCHA | JavaScript | Rate Limit | TLS | robots.txt | Blocks AI | Tier | |
|---|---|---|---|---|---|---|---|---|---|
| Transport & Logistics | 70% | 50% | 10% | 20% | 0% | 20% | 80% | 0% | Tier 2 |
| Airlines | 80% | 10% | 10% | 50% | 20% | 20% | 80% | 20% | Tier 2 |
| Food & Beverages | 100% | 10% | 20% | 80% | 0% | 0% | 80% | 20% | Tier 2 |
| Manufacturing | 70% | 10% | 10% | 50% | 10% | 30% | 60% | 10% | Tier 2 |
| Medical Devices | 90% | 0% | 0% | 40% | 0% | 0% | 90% | 0% | Tier 2 |
| Recruitment & Staffing | 80% | 30% | 30% | 40% | 10% | 30% | 70% | 30% | Tier 2 |
| Retail | 100% | 10% | 10% | 70% | 20% | 20% | 100% | 40% | Tier 2 |
| Sporting Goods | 80% | 10% | 20% | 20% | 10% | 30% | 80% | 0% | Tier 2 |
| Veterinary | 100% | 40% | 20% | 50% | 20% | 20% | 90% | 10% | Tier 2 |
| Visual Art | 100% | 30% | 30% | 30% | 0% | 20% | 100% | 10% | Tier 2 |
| Music | 100% | 30% | 40% | 50% | 20% | 20% | 100% | 20% | Tier 2 |
| Public Safety | 100% | 10% | 40% | 50% | 10% | 20% | 90% | 10% | Tier 2 |
| TV & Film | 100% | 30% | 20% | 30% | 20% | 10% | 80% | 20% | Tier 2 |
| Wholesalers | 90% | 40% | 30% | 30% | 30% | 40% | 80% | 20% | Tier 2 |
| Biotech | 80% | 30% | 50% | 50% | 10% | 20% | 80% | 30% | Tier 2 |
Biotech is the most accessible industry in Japan, averaging mean tier 1.70. Industries ranked by mean recommended browser tier.