AI
Articles from the Zyte blog about AI.

AI web scraping and scalable data collection
AI changes extraction, adaptation and the cost of adding a source. It does not change access. What is genuinely different, with the arithmetic.

Stopping a crawl when the data stops looking real with Jev
A Scrapy pipeline that asks a fast, calibrated AI model whether each scraped field still looks real, and stops the crawl when too many don't. What it caught, what it misses, and whether building it was worth it.

uv Python cheatsheet: what changed in 0.12 and what still trips you up
uv 0.12 changed several defaults that Python developers rely on—from how `uv init` structures projects to how hash checking and lockfiles behave. This practical cheatsheet tests the differences across versions and highlights the command-line traps most likely to surprise you.
Give your AI agent a browser: search, compare, and check product availability with Zyte CDP
What if your AI agent could use a real browser to search, click, compare products, and collect data, without you writing a selector for every step? By connecting it to a remote Zyte CDP browser, you can turn plain-language instructions into practical browser automation.

Four in 10 sites block AI bots with robots.txt
Nearly four in 10 of the world's top sites are now closed to well-behaved AI crawlers. Here's what operators actually do with robots.txt, and which AI agents they block or welcome.
New Zyte add-ons: Agent Skills, Codex, GitHub and more updates
A host of additions to Zyte’s agentic scraping toolbelt helps developers go from prompt to working spider code and beyond.

Harness Engineering #3- Headless mode: the minimal agent harness
Harness engineering part 3: headless mode runs a coding agent without the chat. Point it at a web page and get structured data back in about ten lines.

Claude Fable 5 is the new best model for writing scrapers
We ran nine models, including the new GPT-5.6, through our Zyte Scraping Code Benchmark. Claude Fable 5 puts Sol and others in the shade - but it’s pricey, and the best extraction code still depends on the best infrastructure.

GPT-5.6, Fable 5, and GLM-5.2 entered a bar “crawl” and got hit by The Rate Limit
When OpenAI shipped GPT-5.6 and Anthropic's Fable 5 was sitting at the top of the price list, my question was not "which one wins a leaderboard." It was the one I actually pay for: for the scraping I do, how much model do I need to buy?

Treat your AI skills like software, starting with evals
Most AI skills are never tested — and it shows. Here's how Zyte evaluates scraping skills like real software, catching failures demos miss.

Harness Engineering, part 2: harnessing a data extraction agent
Point an agent at a website, name the fields, get structured records back. Harness engineering part 2: the harness decisions that make a data extraction agent reliable.

Harness Engineering, part 1: what is an agent harness and why it matters
Same model, same weights, zero retraining — LangChain changed nothing but the scaffolding around a coding model and jumped it from 30th place to the top five on a benchmark. That scaffolding has a name: the harness. And it's the part you actually control.