We’ve made a change. Scrapinghub is now Zyte! 

Smart Proxy Manager

(formerly Crawlera)

We manage hundreds of thousands of proxies, so you don't have to

Try free

10yrs

web scraping experience

13bn

pages extracted monthly

10m

records validated per day

99.9%

success rate

The world's preeminent rotating proxy network ensures your web data is delivered quickly and reliably.

You don’t need your own proxies. We can manage thousands of proxies on your behalf.
Quick, easy set up. Just integrate our back-connecting API into your crawler.
Automatic retry handling and rotation with fingerprinting logic to maximize success rates.
Built-in anti-ban management. Compliant web data extraction. Ethically sourced and maintained IPs.

Good to know:

With Smart Proxy Manager you only pay for successful requests delivered.

Results focused

You only pay for successful requests delivered. With no charges for underused IPs or engineer hours spent monitoring and rotating proxies.

Battle tested

Our own 100+ developer team trusts Smart Proxy Manager to deliver accurate, reliable web data to clients including Fortune 500 companies.

We get you

At Zyte we speak your language. Our powerful, easy to use web data extraction and smart rotating proxy solutions are built for developers by developers.

Why Smart Proxy Manager?

“Zyte was able to offer the most simple and effective rotating proxy solution for us. It just works.”

Aurélien Jemma, CEO at Liwango

“It was literally 5 lines of code to get started with Smart Proxy Manager and see crawling success.”

Oskar Bruening, CTO at Peek

“Without Zyte Smart Proxy Manager (formerly Crawlera) our business is not successful.”

Michael Raburn, co-founder of Bridge Below
Python
PHP
Ruby
NodeJS
Java
C#
Curl
import requests
response = requests.get(
    "example.com",
    proxies={
        "http": "http://{}:@proxy.zyte.com:8011/".format(APIKEY),
    }
)
print(response.text)
?php
use GuzzleHttp\Client;

$client = new Client();
$response = $client->get('http://example.com', [
    'proxy' => 'http://APIKEY:@proxy.zyte.com:8011'
]);
echo $response->getBody();
?
require 'curb'
request = Curl::Easy.new("http://example") do |curl|
    curl.proxy_url = "proxy.zyte.com:8011"
    curl.proxypwd = "APIKEY:"
end
request.perform
puts request.body_str
var request = require('request');
request({
    uri: 'http://example.com',
    proxy: 'APIKEY:@proxy.zyte.com:8011'
}, function callback(error, response, body) {
    console.log(body);
});
import java.io.File;
import javax.net.ssl.SSLContext;
import org.apache.http.HttpHeaders;
import org.apache.http.HttpHost;
import org.apache.http.auth.AuthScope;
import org.apache.http.auth.UsernamePasswordCredentials;
import org.apache.http.client.AuthCache;
import org.apache.http.client.CredentialsProvider;
import org.apache.http.client.config.RequestConfig;
import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.client.protocol.HttpClientContext;
import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
import org.apache.http.conn.ssl.TrustSelfSignedStrategy;
import org.apache.http.impl.auth.BasicScheme;
import org.apache.http.impl.client.BasicAuthCache;
import org.apache.http.impl.client.BasicCredentialsProvider;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.message.BasicHeader;
import org.apache.http.ssl.SSLContexts;
import org.apache.http.util.EntityUtils;

public class ClientProxyAuthentication {

    public static void main(String[] args) throws Exception {
        
        // Trust own CA and all self-signed certs
        SSLContext sslcontext = SSLContexts.custom()
                .loadTrustMaterial(new File("/path/to/jre/lib/security/cacerts"),
                                   "changeit".toCharArray(),
                                   new TrustSelfSignedStrategy())
                .build();

        // Allow TLSv1.2 protocol only
        SSLConnectionSocketFactory sslsf = new SSLConnectionSocketFactory(
                sslcontext, new String[] {"TLSv1.2"},
                null,
                SSLConnectionSocketFactory.getDefaultHostnameVerifier());
        
        CredentialsProvider credsProvider = new BasicCredentialsProvider();
        credsProvider.setCredentials(
                new AuthScope("proxy.zyte.com", 8011),
                new UsernamePasswordCredentials("", ""));
        
        try (CloseableHttpClient httpclient = HttpClients.custom()
                .setDefaultCredentialsProvider(credsProvider)
                .setSSLSocketFactory(sslsf)
                .build()) {
            HttpHost target = new HttpHost("example.com", 443, "https");
            HttpHost proxy = new HttpHost("proxy.zyte.com", 8011);

            AuthCache authCache = new BasicAuthCache();

            BasicScheme basicAuth = new BasicScheme();
            basicAuth.processChallenge(
                    new BasicHeader(HttpHeaders.PROXY_AUTHENTICATE,
                                    "Basic realm=\"Zyte\""));
            authCache.put(proxy, basicAuth);

            HttpClientContext ctx = HttpClientContext.create();
            ctx.setAuthCache(authCache);

            RequestConfig config = RequestConfig.custom()
                .setProxy(proxy)
                .build();
            
            HttpGet httpget = new HttpGet("/");
            httpget.setConfig(config);

            try (CloseableHttpResponse response = httpclient.execute(
                target, httpget, ctx)) {
                System.out.println(response.getStatusLine());
                System.out.println(EntityUtils.toString(response.getEntity()));
                EntityUtils.consume(response.getEntity());
            }
        }
    }
}
using System;
using System.IO;
using System.Net;

namespace ProxyRequest
{
    class MainClass
    {
        public static void Main (string[] args)
        {
            var myProxy = new WebProxy("proxy.zyte.com:8011");
            myProxy.Credentials = new NetworkCredential(APIKEY, "");
            
            HttpWebRequest request = (HttpWebRequest)WebRequest.Create("http://example.com");
            request.Proxy = myProxy;
            request.PreAuthenticate = true;
            
            WebResponse response = request.GetResponse();
            Stream dataStream = response.GetResponseStream();
            var reader = new StreamReader(dataStream);

            Console.WriteLine(reader.ReadToEnd());

            reader.Close();
            response.Close();
        }
    }
}
$ curl http://example.com \
      -U APIKEY: \
      -x proxy.zyte.com:8011
Simple back-connecting API

Built by developers, for developers

Smart Proxy Manager gives developers the power to turn their ideas into reality. Fast. Follow our guide to getting started.

Our simple back-connecting API integrates seamlessly into your workflow, taking the hassle out of proxy management and freeing up valuable coding time. So you can focus on delivering more value from web data.

Pricing

Get on board with our great value Starter and Basic plans. Upgrade as your needs grow.

Starter

From
$29
per month
50K requests per month included
Additional requests at $10 per 10K
50 concurrent requests
From
$99
per month
200K requests per month included
Additional requests at $49 per 100K
50 concurrent requests

Advanced

From
$349
per month
2.5M requests per month included
Additional requests at $149 per 1m
200 concurrent requests

Enterprise

From
$999
per month
Custom amount of requests
Custom request concurrency
IP whitelisting
Dedicated account manager
Custom proxy locations
Custom crawling time-window
24/7 priority support
Personalized onboarding
Guaranteed capacity

All packages come with...

24/7 support
10% discount on annual plans
Automatic proxy rotation
Automatic retries
Automatic proxy optimization
Geolocation
Identify as web browser
Reporting and visualizations
Docs and knowledge base
Support and sessions support
Headless browser support
Cookie management
Configurable browser profiles
Non-obvious ban detection
Why Zyte proxy management

Smart proxy rotation service

Highest success rates

Try free

Our industry-leading rotating proxy network is built on our own sophisticated anti-ban algorithms, with request throttling and hundreds of heuristics to ensure industry-leading success rates.

+99.9%
success rate

Smart Proxy Manager scales effortlessly with any project requirement, right up to billions of requests per month. Upgrading is easy at any time. And our crawl consultants are on hand to help you achieve your goals and get the most out of our proxy solutions.

+11 Billion
successful requests per month
We've got the answers

Frequently asked questions

Can I choose IPs in specific locations?

Yes, we can allocate IPs based on the locations you choose. You can otherwise leave the automatic IP proxy rotation down to us wholly to us, where our proprietary algorithms will choose the best IPs to get you the successful requests.

How many proxies do you offer?

It’s in the order of hundreds of thousands. But we have reserves as well depending on the requirements of our customers. We grow the pool as needed.

Do you offer a free trial?

Yes. We offer a free trial of our smart rotating proxy service for 14 days or 10,000 requests (whichever comes first). Customers interested in an Enterprise Free Trial should contact us.

Can I cancel my subscription at any time?

Yes, you may cancel your subscription at any time

I already have proxies. How will Zyte Smart Proxy Manager help me on top of what I'm already doing?

Smart Proxy Manager (formerly Crawlera) is focused on delivering successful responses without any hassle for the end-user. The internal algorithm deals with bans, manages multiple proxy providers, rotates proxies and handles throttling automatically, among other features. It allows your technical team to focus on data only and not proxies.

Can I use Smart Proxy Manager with my own crawler without using Scrapy, Scrapy Cloud or any other Zyte service?

Definitely. Smart Proxy Manager is a standalone service that can be used with any crawler or HTTP client, independently of the rest of the Zyte platform.

How will dedicated residential proxies help my web scraping project?

Dedicated residential proxies are great at helping gather data from domains with stricter thresholds towards bots and tasks that require location-specific IPs. But not all web scraping projects need residential proxies. Most of the times smart rotating datacenter IP proxies work just the same and give the cost benefit. In these cases, all you need is a smart IP rotation service like Zyte Smart Proxy Manager.

What are proxies?

When you start extracting data from the web on a small scale you might not need proxies to make successful requests and get the data. But, as you scale your project because you need to extract more records or more frequently, you will experience issues. Or the site you're trying to reach might display different content depending on the region. So these are the two cases when you need to start using a proxy solution. Learn more about proxies here.