Field notes from the world of data extraction.
Articles, interviews and analysis on how data is gathered, used and fought over — written by the people closest to it.
When LLM Web Scraping isn't Enough to Scale Web Scraping
If LLM web scraping isn't the answer to increasing speed and accuracy in web scraping and overcoming website bans, what is?
Pioneering web scraping technologies and building a developer community, Part 2
High quality web data extracted from localized websites can be tricky and expensive. We share how we solved geolocation issues with Zyte API’s extended geolocation.
Scraping Product Prices Made Easy: Discover the Best Free Tool
Price scraping tools deliver the best prices on the web. Get the best deals with price-parser, our open source library to accurately scrape prices online.
Advanced Zyte API and Page Object techniques for AI Scraping customization
Extract localized data with Zyte API’s extended geolocation
High quality web data extracted from localized websites can be tricky and expensive. We share how we solved geolocation issues with Zyte API’s extended geolocation.
Overcoming Challenges in Scaling Antiban Evasion Systems
Adding AI to the Web Scraping Tech Stack
AI is effective for helping teams scale up the creation and maintenance of web crawlers and extractors making it easy to add new sources to solve additional business cases.
Scraped Data Quality
Making sure your web data is trustworthy when web scraping at scale is incredibly important, so one of the defining success factors is a comprehensive quality assurance (QA) process.
Web Scraping at Scale
Web scraping at scale requires a plan to be successful. You need to define the business case, success metrics, data needed and budget.
Legal Compliance for Web Scraping Project
Operating legal and ethical web scraping projects for legal compliance at scale needs legal oversight by lawyers with a specialization in web scraping.
The New Guide to Web Scraping at Scale
The challenges of scaling web scraping are underestimated. This whitepaper shares what to consider when scaling your web scraping efforts from planning and design, legal compliance, crawling and extracting, and quality assurance.
Key Questions for Scaling Web Scraping
Here are some questions to ask your team and partners that can help you with your web scraping scaling journey.