Zyte Blog
Field notes from the world of data extraction.
Articles, interviews and analysis on how data is gathered, used and fought over — written by the people closest to it.
Browse
Hot topics
Latest
What we've been publishing
Zyte Developer Community Meetup #2: the recap, slides, and recordings
Recordings, slides, and a rundown of Zyte's second Developer Community Meetup with Humanbound: a live prompt injection attack on a price agent, Zyte CDP managed browsers, and the launch of harness-run.
Inside Scrapy's RemoteControl extension
Scrapy 2.19's RemoteControl extension runs a small authenticated HTTP server inside every crawl. How it starts, how it's secured, what the code you send can reach, and what you can do with it on a live crawl, plus the traps and what changes under scrapy-zyte-api.
Scrapy MCP: debug your crawl while it runs
Scrapy MCP connects an AI agent to a running Scrapy crawl, using the new Remote Control extension in Scrapy 2.19 to let the agent discover jobs, check status and run Python against the live crawler, all without restarting it.
One Scrapy Spider, Three Browser Setups: Playwright, Patchright, and Zyte
One Scrapy spider, three browser setups: stock Playwright, Patchright via the new PLAYWRIGHT_BROWSER_PROVIDER hook, and a remote browser on Zyte over CDP. Same spider and selectors throughout, only the browser changes.
Stopping a crawl when the data stops looking real with Jev
A Scrapy pipeline that asks a fast, calibrated AI model whether each scraped field still looks real, and stops the crawl when too many don't. What it caught, what it misses, and whether building it was worth it.
Jev, the model that cannot write a word, and where it fits in web scraping(does it?)
Jev by TypeFace AI cannot generate a string, so it cannot extract a field. What it is, how it differs from an LLM, and the one job it earns in a scraping pipeline.
Inside antibot, the arms race you can't see
Only 18.5% of top sites run dedicated antibot, but every one of them chose to. Why it's the most intentional barrier in the stack, and the strongest signal of a hardened site.
uv Python cheatsheet: what changed in 0.12 and what still trips you up
uv 0.12 changed several defaults that Python developers rely on—from how `uv init` structures projects to how hash checking and lockfiles behave. This practical cheatsheet tests the differences across versions and highlights the command-line traps most likely to surprise you.
Firewalls run the web, just not on purpose
Web Application Firewalls run on 92.4% of the top websites, but most arrived bundled with a CDN and were never tuned to resist automated access.
.png&w=3840&q=75)
Give your AI agent a browser: search, compare, and check product availability with Zyte CDP
What if your AI agent could use a real browser to search, click, compare products, and collect data, without you writing a selector for every step? By connecting it to a remote Zyte CDP browser, you can turn plain-language instructions into practical browser automation.
The page your agent scrapes is now an attack surface. Is it ready for the hostile web?
Real web is becoming hostile for AI Agents. The page your agent scrapes now could be a potential attack surface. Read more and join Zyte's virtual meet-up to see it in action.
Running Playwright at scale: connecting to the Zyte CDP browser
We've released our CDP Browser, ideal for those with existing Playwright scripts looking for better access, as well as developers who need granular control over a browser, but don't want the hassle of running locally.
Bluesky
X
Mastodon
LinkedIn
Facebook
dev.to
RSS feedZyte YouTube
Watch & learn
Video · Zyte YouTube
AI generated these Scrapy projects - why I won't ship them
July 7, 2026
Video · Zyte YouTube
Screenshot webpages with this Claude Skill and Zyte API
March 6, 2026
Video · Zyte YouTube
0% Hallucination? RAG + Web Scraping (Step-by-Step)
March 5, 2026
Video · Zyte YouTube
Generate HTML Parsing code the right way with Scrapy & Web Scraping Copilot
February 23, 2026
Video · Zyte YouTube
Zyte API Sessions - flexible cookie management maintaining control
February 6, 2026
Events & community
Where Zyte shows up
7–8 Oct 2026Extract Summit 2026 — Austin, TX
In-person conference · Austin, TX
10–11 Nov 2026Extract Summit 2026 — Dublin
In-person conference · Dublin, Ireland
On demandShip Agents That Survive the Real Web
Webinar · Zoom, with Humanbound
On demand2026 Web Scraping Industry Report by Zyte
Webinar · On demand