Field notes from the world of data extraction.
Articles, interviews and analysis on how data is gathered, used and fought over — written by the people closest to it.

How to transfer browser cookies to an http session when web scraping

Web data for scraping developers in 2026: AI fuels the agentic future
AI is reshaping web scraping in 2026. Learn how agentic systems, scraping APIs, and automated access management help developers build faster, more resilient data pipelines.

Web data for engineering leaders in 2026: Scale scraping without scaling headcount
Explore how agentic AI is transforming web scraping in 2026, and why engineering leaders should rethink DIY infrastructure.

How to create a Docker container with Scrapy and PostgreSQL

The future I dreamed of is dawning
Zyte's CEO Shane Evans shares a 15-year vision for effortless, AI-driven web data extraction and introduces the 2026 Web Scraping Industry Report with 26 actionable insights.

2026 Web Scraping Industry Report.
View the free 2026 Web Scraping Industry Report and learn how modern teams use AI and automation to collect reliable, compliant web data.

Best proxy providers for web scraping in 2026 | Zyte
Compare the best proxy providers for web scraping in 2026. Learn which residential, ISP, and mobile proxies work best—and when teams move beyond proxies to automation.

Swiss Army Knife Docker Container for Web Scraping

Scrapy in 2026: New release brings modern async crawling standards
Scrapy 2.14.0 modernizes the framework with native async/await, smarter scheduling, and cleaner spider configuration. Here’s what the new release means for production crawlers in 2026.

Building blocks of a web scraping business
Why most web scraping startups fail—and how to build one that scales. Learn real unit economics, proxy costs, margins, and a path to SaaS-level profitability.

Data in 2026: Bets and forecasts from web experts
What will 2026 have in store for the world of web data? We asked eight experts for their predictions.

AI and the web: What 2025 changed and what comes next
2025 was the year AI learned to reason. From reasoning-first LLMs to autonomous agents and a reshaped web economy, this retrospective explores what changed—and what’s coming next.