Developer to Developer consultation for top challenges

On-demand packages by a team of Zyte web scraping experts. 12+ years of experience delivering 13bn pages a month.

Developer-to-developer knowledge transfer

Technical onboarding

Clients interested in migrating to Zyte API can benefit from advisory services to ensure a smooth migration.

  • Basic orientation in Zyte API & Scrapy Cloud - highlighting key features and how to get started

  • Zyte API & Scrapy Cloud usage examples - how to use our APIs effectively

  • Troubleshooting processes - some basic tips on how to approach troubleshooting; e.g. checking if any anti-ban has been introduced or changed

David Comber, Solutions Architecture Manager

Scaling Up - Essentials

Learn coding best practices for web scraping. Training in Python code style best practices - using tools like Black, Isort, Pre-commit, following PEP8 style, and using linters to ensure code adheres to industry standards and best practices.

  • Setup standardized tools -setup Black, Isort, pre-commit, Scrapy Cloud and Spidermon

  • Introduction to Spidermon - overview of Spidermon and basic monitoring

  • Workshop on Spidermon -usage, error checking, email notification, sentry (or similar), Scrapy stats, reporting and advanced monitoring

Victor Ruiz, Technical Practices Lead

Scaling Up - Expert Review

A thorough analysis of your web scraping solution to identify potential points for optimization.

  • Audit of current web scraping - walkthrough of code and a full review of Crawling Strategy and Infrastructure

  • Report on audit findings to identify primary bottlenecks (e.g. velocity, parallel processes, etc.). Identify inconsistencies, risks, and cost-saving recommendations

  • Workshop on audit findings to optimize your web scraping solution

Victor Ruiz, Technical Practices Lead

Data Quality - Essentials

Introduction to Data Quality Assurance and automated & manual data quality practices

  • Schema Validation - turning the inherent structured nature of your data into fast, whole-dataset validation per data point

  • Advanced data quality assurance with Python. Take data quality assurance further with advanced techniques to uncover issues and anomalies

  • Visualizing Data Quality - Turn advanced data quality assurance into charts and visualisations to highlight trends and tell the data quality story

Evan Phelan, Head of QA

Data Quality - Expert Review

A thorough analysis of your current Data Quality Assurance setup to identify potential opportunities for improvement.

  • Audit of current Data Quality setup. Walkthrough of code and tools accompanied by Zyte QA Engineer. Review of monitoring strategy AND response capabilities

  • Report on audit findings to identify processes and tooling inefficiencies, risks, and Data  Quality improvement recommendations

  • Workshop on audit findings to optimize your data quality solution

Evan Phelan, Head of QA