Explore resources by topic or category

Heads up, Kimono Labs users!
Today, we are releasing a tool to help you migrate your Kimono projects to Portia.

Scrapy with beta Python 3 support is finally here! Released through Scrapy 1.1.0rc1, this is the result of several months of hard work on the part of the Scrapy community and Zyte engineers.

Today we released the latest version of Portia bringing with it the ability to crawl pages that require JavaScript. To celebrate this release we are making Splash available as a free trial to all Portia users so you can try it out with your projects.

This past year, we have been working on a distributed version of our crawl frontier framework, Frontera. This work was partially funded by DARPA and is included in the DARPA Open Catalog.

EuroPython 2015 is happening this week and we’re having the largest company meetup so far as a part of it, with more than 30 members from our fully remote-working team attending.

We are proud to announce some exciting changes we've introduced this week. These changes bring a much more pleasant user experience, and several new features including the addition of Portia to our platform!

We’re proud to announce our new open source project, ScrapyRT! ScrapyRT, short for Scrapy Real Time, allows you to extract data from a single web page via an API using your existing Scrapy spiders.

One of the things that takes more time when building a spider is reviewing the scraped data and making sure it conforms to the requirements and expectations of your client or team.

We're excited to introduce Dash, a major update to our scraping platform.
This release is the final step in migrating to our new storage back end and contains improvements to almost every part of our infrastructure. In this post I'd like to introduce some of the highlights.

We are proud to introduce Zyte Smart Proxy Manager , a smart web downloader designed specifically for web crawling.

After a year considering it, we have decided to go ahead and drop support for Python 2.5 in Scrapy. Starting from 0.15, Scrapy will require Python 2.6 or above.