zyte logo

Smart Proxy Manager

(formerly Crawlera)

Our rotating proxy service manages hundreds of thousands of proxies, so you don't have to

Try free

What is Smart Proxy Manager?

When extracting web data at scale using proxy management is critical to avoid getting banned or blocked. Smart Proxy Manager automatically selects the best proxies to keep your crawl healthy. It handles retries and applies rotation and fingerprinting logic to maximize your success rate.

The world's preeminent rotating proxy network ensures your web data is delivered quickly and reliably. You don’t need your own proxies. We manage thousands of proxies on your behalf.

Smart Proxy Manager features

  • Automatic proxy rotation
  • Automated retries and replacement of blocked IPs ensure the highest success rates
  • Automatic retries
  • Handles retries on your blocked requests to maximise success rates
  • Ban detection
  • Built-in, extensive, ever-growing database of bans for automatic ban detection.
  • Geolocation
  • Extensive datacenter proxy coverage for over 50 countries
  • Session support
  • Manage your own sessions so you can send multiple requests through the same IP.
  • Mimic real browser
  • Use browser profiles to appear like different browsers and TLS fingerprints to minimize bans, optionally overriding them with your own.
  • Headless browser support
  • Integrate your headless browser scraping tool easily with our client libraries for Puppeteer, Playwright, and Selenium.

  • Powerful insights
  • Visualize the performance of your crawler in a dashboard. Get reports and improvement recommendations.

Try Zyte's rotating proxy service for free

With Smart Proxy Manager you only pay for successful requests delivered.

Built for developers by developers

Integrations with Smart Proxy Manager

Smart Proxy Manager gives developers the power to turn their ideas into reality. Fast. Follow our guide to getting started.

Our simple back-connecting API integrates seamlessly into your tech stack, taking the hassle out of proxy management and freeing up valuable coding time. So you can focus on delivering more value from web data.

 

Python
PHP
Ruby
NodeJS
Java
C#
Curl
import requests
response = requests.get(
    "example.com",
    proxies={
        "http": "http://{}:@proxy.zyte.com:8011/".format(APIKEY),
    }
)
print(response.text)
?php
use GuzzleHttp\Client;

$client = new Client();
$response = $client->get('http://example.com', [
    'proxy' => 'http://APIKEY:@proxy.zyte.com:8011'
]);
echo $response->getBody();
?
require 'curb'
request = Curl::Easy.new("http://example") do |curl|
    curl.proxy_url = "proxy.zyte.com:8011"
    curl.proxypwd = "APIKEY:"
end
request.perform
puts request.body_str
var request = require('request');
request({
    uri: 'http://example.com',
    proxy: 'APIKEY:@proxy.zyte.com:8011'
}, function callback(error, response, body) {
    console.log(body);
});
import java.io.File;
import javax.net.ssl.SSLContext;
import org.apache.http.HttpHeaders;
import org.apache.http.HttpHost;
import org.apache.http.auth.AuthScope;
import org.apache.http.auth.UsernamePasswordCredentials;
import org.apache.http.client.AuthCache;
import org.apache.http.client.CredentialsProvider;
import org.apache.http.client.config.RequestConfig;
import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.client.protocol.HttpClientContext;
import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
import org.apache.http.conn.ssl.TrustSelfSignedStrategy;
import org.apache.http.impl.auth.BasicScheme;
import org.apache.http.impl.client.BasicAuthCache;
import org.apache.http.impl.client.BasicCredentialsProvider;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.message.BasicHeader;
import org.apache.http.ssl.SSLContexts;
import org.apache.http.util.EntityUtils;

public class ClientProxyAuthentication {

    public static void main(String[] args) throws Exception {
        
        // Trust own CA and all self-signed certs
        SSLContext sslcontext = SSLContexts.custom()
                .loadTrustMaterial(new File("/path/to/jre/lib/security/cacerts"),
                                   "changeit".toCharArray(),
                                   new TrustSelfSignedStrategy())
                .build();

        // Allow TLSv1.2 protocol only
        SSLConnectionSocketFactory sslsf = new SSLConnectionSocketFactory(
                sslcontext, new String[] {"TLSv1.2"},
                null,
                SSLConnectionSocketFactory.getDefaultHostnameVerifier());
        
        CredentialsProvider credsProvider = new BasicCredentialsProvider();
        credsProvider.setCredentials(
                new AuthScope("proxy.zyte.com", 8011),
                new UsernamePasswordCredentials("", ""));
        
        try (CloseableHttpClient httpclient = HttpClients.custom()
                .setDefaultCredentialsProvider(credsProvider)
                .setSSLSocketFactory(sslsf)
                .build()) {
            HttpHost target = new HttpHost("example.com", 443, "https");
            HttpHost proxy = new HttpHost("proxy.zyte.com", 8011);

            AuthCache authCache = new BasicAuthCache();

            BasicScheme basicAuth = new BasicScheme();
            basicAuth.processChallenge(
                    new BasicHeader(HttpHeaders.PROXY_AUTHENTICATE,
                                    "Basic realm=\"Zyte\""));
            authCache.put(proxy, basicAuth);

            HttpClientContext ctx = HttpClientContext.create();
            ctx.setAuthCache(authCache);

            RequestConfig config = RequestConfig.custom()
                .setProxy(proxy)
                .build();
            
            HttpGet httpget = new HttpGet("/");
            httpget.setConfig(config);

            try (CloseableHttpResponse response = httpclient.execute(
                target, httpget, ctx)) {
                System.out.println(response.getStatusLine());
                System.out.println(EntityUtils.toString(response.getEntity()));
                EntityUtils.consume(response.getEntity());
            }
        }
    }
}
using System;
using System.IO;
using System.Net;

namespace ProxyRequest
{
    class MainClass
    {
        public static void Main (string[] args)
        {
            var myProxy = new WebProxy("proxy.zyte.com:8011");
            myProxy.Credentials = new NetworkCredential(APIKEY, "");
            
            HttpWebRequest request = (HttpWebRequest)WebRequest.Create("http://example.com");
            request.Proxy = myProxy;
            request.PreAuthenticate = true;
            
            WebResponse response = request.GetResponse();
            Stream dataStream = response.GetResponseStream();
            var reader = new StreamReader(dataStream);

            Console.WriteLine(reader.ReadToEnd());

            reader.Close();
            response.Close();
        }
    }
}
$ curl http://example.com \
      -U APIKEY: \
      -x proxy.zyte.com:8011

The industry-leading rotating proxy service

10 years
web scraping experience
13bn
pages extracted monthly
10m
records validated per day
99.9%
success rate

Smart Proxy Manager prices

Get on board with our great value Starter and Basic plans. Upgrade as your needs grow.

Starter

From $29/month
  • 50K requests per month included
  • Additional requests at $10 per 10K
  • 50 concurrent requests

Basic

From $99/month
  • 200K requests per month included
  • Additional requests at $49 per 100K
  • 50 concurrent requests

Advanced

From $349/month
  • 2.5M requests per month included
  • Additional requests at $149 per 1m
  • 200 concurrent requests

Enterprise

From $999/month
  • Custom amount of requests
  • Custom request concurrency
  • IP whitelisting
  • Dedicated account manager
  • Custom proxy locations
  • Custom crawling time-window
  • 24/7 priority support
  • Personalized onboarding
  • Guaranteed capacity

All packages come with...

  • 24/7 support
  • 10% discount on annual plans
  • Automatic proxy rotation
  • Automatic retries
  • Automatic proxy optimization
  • Geolocation
  • Identify as web browser
  • Reporting and visualizations
  • Docs and knowledge base
  • Support and sessions support
  • Headless browser support
  • Cookie management
  • Configurable browser profiles
  • Non-obvious ban detection
  • Residential IPs (add-on)

Smart Proxy Manager - Residential IP service

Besides our datacenter proxies, Smart Proxy Manager offers residential proxies. Use the Residential add-on in your active Smart Proxy Manager subscription for geo-blocked websites. 

Learn more

Smart Proxy Manager use cases

Get the most of our rotating proxy service. And see what others can do with Smart Proxy Manager.

Liwango: Powering vacation rental intelligence with Smart Proxy Manager

Read more

Peek: Enabling 30 million requests/month with Smart Proxy Manager 

Read more
Residential proxies by Zyte

Bridge Below: Providing a smart proxy solution to extract automotive web data

Read more

Smart Proxy Manager reviews

Check out what our customers say about Smart Proxy Manager

“Zyte was able to offer the most simple and effective rotating proxy solution for us. It just works.”

Aurélien Jemma, CEO at Liwango

“It was literally 5 lines of code to get started with Smart Proxy Manager and see crawling success.”

Oskar Bruening, CTO at Peek

“Without Zyte Smart Proxy Manager  our business is not successful.”

Michael Raburn, co-founder of Bridge Below
We've got the answers

Rotating proxy FAQs

Can I choose IPs in specific locations?

Yes, we can allocate IPs based on the locations you choose. You can otherwise leave the automatic IP proxy rotation down to us wholly to us, where our proprietary algorithms will choose the best IPs to get you the successful requests.

How many proxies do you offer?

It’s in the order of hundreds of thousands. But we have reserves as well depending on the requirements of our customers. We grow the pool as needed.

Do you offer a free trial?

Yes. We offer a free trial of our smart rotating proxy service for 14 days or 10,000 requests (whichever comes first). Customers interested in an Enterprise Free Trial should contact us.

Can I cancel my subscription at any time?

Yes, you may cancel your subscription at any time

I already have proxies. How will Zyte Smart Proxy Manager help me on top of what I'm already doing?

Smart Proxy Manager (formerly Crawlera) is focused on delivering successful responses without any hassle for the end-user. The internal algorithm deals with bans, manages multiple proxy providers, rotates proxies and handles throttling automatically, among other features. It allows your technical team to focus on data only and not proxies.

Can I use Smart Proxy Manager with my own crawler without using Scrapy, Scrapy Cloud or any other Zyte service?

Definitely. Smart Proxy Manager is a standalone service that can be used with any crawler or HTTP client, independently of the rest of the Zyte platform.

What are proxies?

When you start extracting data from the web on a small scale you might not need proxies to make successful requests and get the data. But, as you scale your project because you need to extract more records or more frequently, you will experience issues. Or the site you're trying to reach might display different content depending on the region. So these are the two cases when you need to start using a proxy solution. Learn more about proxies here.