Zyte Blog

Field notes from the world of data extraction.

Articles, interviews and analysis on how data is gathered, used and fought over — written by the people closest to it.

⌕

Access handling

The State of Web Access: How today's web raises its barriers

The largest ever audit of web access control mechanisms lifts the lid on how sites are regulating programmatic visits.

Robert AndrewsAugust 5, 2026

Browser automation scripts - Puppeteer and Playwright

Product Update

Introducing Zyte CDP support: Your browser automation, our infrastructure

Valter Sciarrillo

Web data collection

Data is the new water

Robert Andrews10 min read

Browse

Hot topics

AI-assisted data extraction Data quality Scraping strategy Data gathering for AI Web scraping APIs

Latest

What we've been publishing

See all

Article

Give your AI agent a browser: search, compare, and check product availability with Zyte CDP

What if your AI agent could use a real browser to search, click, compare products, and collect data, without you writing a selector for every step? By connecting it to a remote Zyte CDP browser, you can turn plain-language instructions into practical browser automation.

John RooneyAISeptember 11, 2026

Ernesto Hilvano smiles to camera on a video call, with a Pikuball match playing behind him: two anime-style players on a teal court in a blossom-filled garden, the scoreboard reading Kristianst 0, Tomokawata 2.

Article

What building an open-world pickleball MMO for my unborn baby taught me about infrastructure scalability

It was just a silly game side project. But, as I grew my sim to 3,000 matches, I levelled up on edge routing, flattening memory spikes and hardcore monitoring.

ErnestTool-assisted codingSeptember 9, 2026

Article

The page your agent scrapes is now an attack surface. Is it ready for the hostile web?

Real web is becoming hostile for AI Agents. The page your agent scrapes now could be a potential attack surface. Read more and join Zyte's virtual meet-up to see it in action.

Ayan PahwaSeptember 8, 2026

Article

Running Playwright at scale: connecting to the Zyte CDP browser

We've released our CDP Browser, ideal for those with existing Playwright scripts looking for better access, as well as developers who need granular control over a browser, but don't want the hassle of running locally.

John RooneyHow ToSeptember 7, 2026

A toy robot walking toward a blocking robots.txt file

Article

Four in 10 sites block AI bots with robots.txt

Nearly four in 10 of the world's top sites are now closed to well-behaved AI crawlers. Here's what operators actually do with robots.txt, and which AI agents they block or welcome.

Robert AndrewsData gathering for AISeptember 7, 2026

Article

Fable 5.1 shipped, and GLM-5.3-Flash turned out to be someone I already met

A personal take on Claude Fable 5.1 and GLM-5.3-Flash: real benchmarks, a live extraction test, and the model I guessed before Zhipu confirmed it.

Ayan PahwaSeptember 2, 2026

Article

Web data in a reactive notebook: an introduction to marimo

marimo is a reactive Python notebook that reruns only affected cells. Learn to scrape web data with Zyte API, chart prices, and cache costly API calls.

Ayan PahwaSeptember 2, 2026

Article

75% of the web uses robots.txt - here's how

Three in four of the world's top sites publish a robots.txt, yet few name individual crawlers. A look inside the web's advisory layer: coverage, sophistication and limits.

Robert AndrewsFuture of the webSeptember 2, 2026

Article

Introducing Zyte CDP support: Your browser automation, our infrastructure

Want to use traditional browser automation frameworks without infrastructure headaches? Tap our new CDP support to run scripts on Zyte’s powerful infrastructure.

Valter SciarrilloProduct UpdateSeptember 1, 2026

Text file struggling under the weight of European privacy burden.

Article

Why Europe’s new AI scraping guidelines miss the mark

New guidelines on generative AI scraping aim to protect user privacy. But turning a rudimentary, 30-year-old web server standard into a legal barrier will disenfranchise users and create accidental monopolies.

Sally-Anne HinfeyWeb data collection legalityAugust 25, 2026

Article

WebFetch is ‘lossy’ by design: give your coding agent a better fetch in one command

Your coding agent built-in webfetch tool is not the best and it's hampering your research and coding workflows, fix it with one CLI tool and never face blocks again.

Ayan PahwaAugust 25, 2026

Article

Revealed: How retailers use tech to block the bots

The web’s big general marketplaces exist to be browsed by everyone. But they are also the second most defended sector online, shutting the door on AI crawlers.

Robert AndrewsAccess handlingAugust 24, 2026

More articles More videos More webinars More case studies More white papers