Overview
JavaScript is Hong Kong SAR's most common secondary barrier
| Value | |
|---|---|
| WAF | 91.8% |
| Antibot | 20.4% |
| CAPTCHA | 21.4% |
| JavaScript | 42.1% |
| Rate Limiting | 13.8% |
| TLS | 17.6% |
WAF dominates at 91.8%. Among secondary barriers, JavaScript leads at 42.1% while Rate Limiting sits lowest at 13.8%, creating a pronounced gap between the top and bottom of the radar.
Barrier Landscape
JavaScript is the lead secondary barrier
| Value | |
|---|---|
| WAF | 91.8% |
| Antibot | 20.4% |
| CAPTCHA | 21.4% |
| JavaScript | 42.1% |
| Rate Limiting | 13.8% |
| TLS | 17.6% |
| Has robots.txt | 78.4% |
| Blocks AI crawlers | 14.4% |
WAF is near-universal at 91.8%. JavaScript, the top secondary barrier, reaches 42.1%. CAPTCHA and Antibot cluster close behind at 21–21%, while Rate Limiting trails at 13.8%.
Two-barrier sites are the most common configuration
| % of sites | |
|---|---|
| 0 barriers | 5.8% |
| 1 barrier | 30.5% |
| 2 barriers | 33.4% |
| 3 barriers | 16.8% |
| 4 barriers | 8.9% |
| 5 barriers | 4.1% |
| 6 barriers | 0.5% |
A third of Hong Kong SAR sites (33.4%) carry exactly two barriers. Single-barrier sites account for 30.5%, and 30.3% stack three or more. Only 5.8% present no barrier at all.
Most Hong Kong SAR sites are Easy to access
| Mean recommended tier | |
|---|---|
| All sites | 1.7 |
Browser-based retrieval is recommended for most sites. The mean tier of 1.7 reflects a concentration at Tier 1, while the median of 2 shows that fewer than half of sites resolve at the simplest level.
Nearly nine in ten sites are Simple or Easy to access
| % of sites | |
|---|---|
| 1 - Simple | 45.8% |
| 2 - Easy | 41.2% |
| 3 - Moderate | 10.5% |
| 4 - Complex | 2% |
| 5 - Advanced | 0.6% |
Tier 1 accounts for 45.8% of sites and Tier 2 for 41.2%, together covering 87% of the sample. Tier 3 adds 10.5%. Tiers 4 and 5 are minimal at 2.0% and 0.6% respectively.
robots.txt
AI crawlers are most cited, ahead of SEO audit tools
| % of sites | |
|---|---|
| AI crawlers | 15.9% |
| SEO / search indexing | 12% |
| SEO audit tools | 14.5% |
| Commercial scrapers | 8.4% |
| Social / content scrapers | 5.8% |
| Archive / research crawlers | 4.6% |
| Monitoring / uptime bots | 0.6% |
Of all 1,110 Hong Kong SAR sites, 15.9% name at least one AI crawler in robots.txt — the top category. SEO audit tools follow at 14.5%, with SEO/search indexing bots at 12.0%.
Uniform rules are Hong Kong SAR's most common robots.txt posture
| % of sites | |
|---|---|
| Uniform per-agent rules (no path mixing) | 41.4% |
| Path-level mixed rules for same agent | 32.3% |
| Allow-only (no disallow rules) | 2.1% |
| Mixed AI posture (block some, allow others) | 2.3% |
| Blanket Disallow: / | 2.6% |
Uniform per-agent rules apply across 41.4% of all sites. Path-level mixed rules cover 32.3%. The 21.6% of sites with no robots.txt at all represent the largest single unmanaged posture group.
GPTBot faces the most Disallow rules in Hong Kong SAR's robots.txt
| Disallowed | Allowed (whitelisted) | |
|---|---|---|
| GPTBot | 12.2% | 2.2% |
| CCBot | 9% | 1.1% |
| ClaudeBot | 8.9% | 1.1% |
| Google-Extended | 8.7% | 1.5% |
| Bytespider | 7.2% | 0.4% |
| meta-externalagent | 3.2% | 0.1% |
| ChatGPT-User | 6.7% | 2.6% |
| PerplexityBot | 6.6% | 1.9% |
| Applebot-Extended | 4.9% | 0.3% |
| OAI-SearchBot | 4.6% | 1.9% |
| anthropic-ai | 5.6% | 0.9% |
12.2% of all Hong Kong SAR sites carry a Disallow for GPTBot, the highest block rate of any AI agent. ChatGPT-User receives the most explicit Allow rules at 2.6%.
Seven sectors name bots on every site
| % of sites | |
|---|---|
| Agriculture | 100% |
| Computer & Video Games | 100% |
| Consumer Electronics | 100% |
| Design | 100% |
| Entertainment | 100% |
| Retail | 100% |
| Visual Art | 100% |
| Adult | 90% |
| Airlines | 90% |
| Apparel & Fashion | 90% |
| Automotive | 90% |
| Beauty & Cosmetics | 90% |
| Comics & Animation | 90% |
| Food & Beverages | 90% |
| General | 90% |
Agriculture, Computer & Video Games, Consumer Electronics, Design, Entertainment, Retail, and Visual Art all show every measured site naming at least one bot to restrict. A further eight sectors sit at 90%.
Seven sectors share the top explicit bot Allow rate
| % of sites | |
|---|---|
| Airlines | 70% |
| Architecture | 70% |
| Photography | 70% |
| Publishing | 70% |
| Real Estate | 70% |
| Retail | 70% |
| Veterinary | 70% |
| Automotive | 60% |
| Facilities Services | 60% |
| Furniture | 60% |
| Graphic Design | 60% |
| Non-Profit | 60% |
| Plastics & Polymers | 60% |
| Science | 60% |
| Software & Dev | 60% |
Airlines, Architecture, Photography, Publishing, Real Estate, Retail, and Veterinary each have 70% of their sites naming at least one bot in an explicit Allow rule — the highest Allow rate in this market.
Industry Patterns
Furniture is the hardest sector to access programmatically
| WAF | Antibot | CAPTCHA | JavaScript | Rate Limit | TLS | robots.txt | Blocks AI | Tier | |
|---|---|---|---|---|---|---|---|---|---|
| Furniture | 100% | 60% | 30% | 40% | 20% | 70% | 60% | 10% | Tier 3 |
| Jewellery & Luxury | 100% | 20% | 20% | 30% | 40% | 60% | 60% | 0% | Tier 3 |
| Real Estate | 90% | 20% | 20% | 40% | 30% | 20% | 70% | 40% | Tier 3 |
| Equipment & Supplies | 90% | 40% | 10% | 60% | 30% | 10% | 60% | 10% | Tier 3 |
| Computer Hardware | 90% | 10% | 0% | 20% | 20% | 20% | 80% | 10% | Tier 2 |
| Hospitality | 80% | 30% | 0% | 50% | 20% | 40% | 60% | 10% | Tier 2 |
| Retail | 100% | 10% | 0% | 70% | 10% | 10% | 100% | 50% | Tier 2 |
| Human Resources | 100% | 70% | 60% | 40% | 40% | 20% | 70% | 10% | Tier 2 |
| Restaurants | 80% | 40% | 10% | 40% | 20% | 10% | 70% | 10% | Tier 2 |
| Beauty & Cosmetics | 80% | 30% | 20% | 30% | 30% | 30% | 90% | 20% | Tier 2 |
| Apparel & Fashion | 90% | 20% | 30% | 60% | 20% | 30% | 90% | 10% | Tier 2 |
| Consumer Electronics | 80% | 10% | 20% | 40% | 10% | 40% | 100% | 20% | Tier 2 |
| Medical Devices | 90% | 0% | 10% | 50% | 10% | 0% | 90% | 10% | Tier 2 |
| Recruitment & Staffing | 90% | 40% | 50% | 20% | 10% | 30% | 80% | 10% | Tier 2 |
| Sporting Goods | 80% | 10% | 20% | 30% | 30% | 20% | 50% | 0% | Tier 2 |
Furniture leads with a mean recommended tier of 2.80. Jewellery & Luxury and Real Estate both sit at 2.70. The top five sectors all carry tier medians of 3, indicating Moderate scraping complexity is common at the top.
Music, Automotive, Crafts, and Manufacturing share the most accessible floor
| WAF | Antibot | CAPTCHA | JavaScript | Rate Limit | TLS | robots.txt | Blocks AI | Tier | |
|---|---|---|---|---|---|---|---|---|---|
| Textiles & Nonwovens | 100% | 0% | 30% | 40% | 0% | 20% | 100% | 10% | Tier 2 |
| Wine & Spirits | 100% | 30% | 10% | 40% | 20% | 10% | 50% | 30% | Tier 2 |
| Food & Beverages | 100% | 60% | 30% | 60% | 40% | 0% | 90% | 40% | Tier 2 |
| Warehousing | 100% | 20% | 30% | 70% | 20% | 0% | 80% | 20% | Tier 2 |
| Banking | 100% | 30% | 10% | 40% | 10% | 10% | 70% | 0% | Tier 2 |
| Customer Services | 100% | 10% | 20% | 50% | 20% | 20% | 60% | 0% | Tier 2 |
| Design | 100% | 20% | 30% | 40% | 20% | 30% | 100% | 30% | Tier 2 |
| Mail & Delivery | 70% | 40% | 10% | 10% | 0% | 20% | 80% | 0% | Tier 2 |
| Maritime | 90% | 10% | 0% | 50% | 10% | 30% | 90% | 10% | Tier 2 |
| Public Safety | 100% | 40% | 40% | 60% | 20% | 40% | 90% | 10% | Tier 2 |
| Transport & Logistics | 70% | 50% | 10% | 20% | 0% | 20% | 80% | 0% | Tier 2 |
| Automotive | 90% | 10% | 0% | 20% | 40% | 20% | 90% | 40% | Tier 2 |
| Crafts | 100% | 20% | 10% | 50% | 20% | 10% | 70% | 10% | Tier 2 |
| Manufacturing | 80% | 10% | 10% | 30% | 0% | 40% | 70% | 0% | Tier 2 |
| Music | 100% | 40% | 30% | 70% | 30% | 20% | 80% | 10% | Tier 2 |
At the accessible end, Music, Automotive, Crafts, and Manufacturing all post a mean tier of 1.90 — the lowest in the sample. Most sectors in this group cluster at a tier median of 2.