Field notes from the world of data extraction.
Articles, interviews and analysis on how data is gathered, used and fought over — written by the people closest to it.
Why Europe’s new AI scraping guidelines miss the mark
New guidelines on generative AI scraping aim to protect user privacy. But turning a rudimentary, 30-year-old web server standard into a legal barrier will disenfranchise users and create accidental monopolies.
WebFetch is ‘lossy’ by design: give your coding agent a better fetch in one command
Your coding agent built-in webfetch tool is not the best and it's hampering your research and coding workflows, fix it with one CLI tool and never face blocks again.
Revealed: How retailers use tech to block the bots
The web’s big general marketplaces exist to be browsed by everyone. But they are also the second most defended sector online, shutting the door on AI crawlers.
The web is being priced, not blocked
Zyte's State of Web Access research finds new barriers making web scraping more difficult. We interview the researcher who says that doesn't mean the opportunity is over.
The State of Web Access
View the free State of Web Access report to learn just how difficult modern web scraping has become.
Fashion websites are the hardest to size up
While publishers fight over AI crawlers, fashion has built some of the most heavily defended shop windows on the web. So, which controls are in fashion, in fashion?
The web is synthetic. Now look at the human trail
Domagoj Marić explores how AI, web scraping, and OSINT turn scattered personal data into profiles, scams, and security risks at Extract Summit.
The State of Web Access: How different industries behave to bots
Zyte's large-scale audit of web access controls shows how different kinds of businesses exhibit different policies.
A $70-a-month production app replaced a $5,000-a-month platform: What production-grade AI coding looks like
See how Fran Muñoz used production-grade AI coding, specification-driven development, and 3,600 tests to replace a $5,000-a-month platform with a $70 app.
Podcast Episode 10 is out: Skills, packages & updates
Episode 10 of the Zyte podcast is out, and it is one of the widest-ranging conversations we have recorded this year. John Rooney sits down with Neha Setia Nagpal and Ayan Pahwa for an unfiltered, all-in-style chat covering everything the team has been building, reading, and arguing about lately.
Chrome has a new potential fingerprint vector
navigator.cpuPerformance is coming to Chrome in August, v152. What does it mean for fingerprinting and will it mean changes to scraping stacks?
Harness Engineering, part 4: giving your agent a custom fetch tool that survives the real web
Your AI agent is as powerful as the tools it has access to. Here's a tutorial on how you can create your own custom agent tool using Claude Agent SDK for zyte which makes getting structured data from web a breeze.