Field notes from the world of data extraction.
Articles, interviews and analysis on how data is gathered, used and fought over — written by the people closest to it.

Gemini 3.0 Pro Code Gen With Web Scraping Copilot

Build Your Own Price Drop Notifier (Zyte Auto Extract + IFTTT Mobile Alerts)

Hybrid Scraping: The Architecture for the Modern Web

Modern Web Scraping starts with THIS.

3 Rules of Modern Web Scraping

The Modern Scrapy Developer's Guide (Part 3): Auto-Generating Page Objects with the Web Scraping Copilot
In this guide, we'll show you how to use Web Scraping Copilot (our VS Code extension) to automatically write 100% of your Items, Page Objects, and even your unit tests.

The Modern Scrapy Developer's Guide (Part 2): Page Objects with scrapy-poet
In this guide, we'll fix this by refactoring our spider to a professional, modern standard using Scrapy Items and Page Objects (via crapy-poet). We will completely separate our crawling logic from our parsing logic.

The Modern Scrapy Developer's Guide (Part 1): Building Your First Spider
In this definitive guide, we will walk you through, step-by-step, how to build a real, multi-page crawling spider. You will go from an empty folder to a clean JSON file of structured data in about 15 minutes

Zyte’s 2025 review: Year of locks and unlocks
2025 reshaped web scraping. From AI-assisted extraction and escalating bot defenses to clearer legal frameworks and cheaper APIs, Zyte reviews the forces redefining access to web data—and what comes next.

AI’s legal frontier: What Europe’s privacy regulators say about scraping personal data
Explore how EU privacy regulators view AI web scraping, lawful bases like legitimate interest, risks of collecting personal data, and compliance best practices.

Zyte leads the pack in Proxyway’s 2025 Web Scraping API Report
Proxyway’s 2025 Web Scraping API Report ranks Zyte #1 for unblocking success, speed, cost efficiency, and AI-powered data extraction. See the full breakdown.

The Modern Web Scraping Method You NEED to Know
Learn how to scrape data in json format from a websites API