Explore resources by topic or category

In this article, I’llexplain the problem of anti-bot technology for web scraping developers through the lens of the anti-bot distribution curve (a view of the top 250,000 websites and the relative complexity of their anti-bot tech) and the landscape of anti-bot tech across the web.

Zyte’s flagship product, Zyte API, now includes built-in features that automate crawling using spider templates, and our patented AI-powered automated extraction, which gives you quality structured data quickly without writing custom parsing code.

We’re thrilled to announce to our global developer community that Zyte API now comes out of the box with all the features that power our complete solution for scraping with AI, enabling developers to build and launch spiders, unblock websites and extract data from a single UI three times faster than legacy scraping vendors and proxy APIs.

Data mining and web scraping – sounds like two buzzwords meaning the same thing. Quite often data mining is misunderstood as the process of obtaining information from a website;

In the first part, we discussed a template to define the clear purpose of your web scraping system that can help you design your crawlers better and prepare you for the uncertainty involved in a large scale web scraping project.

In this article we discuss some main challenges that e-commerce retailers face on a daily basis due to the amount of web data needed and how to solve them.

After a decade of dedicated service, Smart Proxy Manager (SPM) is being retired for new customers. We’re excited to introduce our flagship Web Scraping API, Zyte API, as its worthy successor — the best and only way to access ethical and industry-leading managed proxy solutions.

Zyte API is the next iteration of Zyte’s best-in class proxy and website unblocking technology. We built it as an HTTP API. This was a conscious design decision: an API to support all your web scraping needs would not work well with the limitations of a proxy API.

In 2023, Meta sued Bright Data for scraping data from Facebook and Instagram, alleging that its scraping breached Facebook and Instagram’s terms of service and is thus a breach of contract.

The reputation of web scraping hasn’t always been the best. Unsavory actors have cast a shadow over the reputable parts of the web scraping industry at large, and it has to stop.

Web scraping developers often find themselves in a struggle to manage bans and blocks. Every time they resolve a ban, it's only a matter of time before their scrapers encounter the same issue again.

Proxyway, one of the most respected proxy provider research blogs, released its in-depth review of five leading proxy APIs on the market. The report, In-Depth Look into Popular Proxy APIs (Web Unblockers), showed that Zyte API is by far the best option when looking at the three major fundamentals: success rate, speed, and cost.