Smart Proxy Manager gives developers the power to turn their ideas into reality. Fast. Follow our guide to getting started.
Our simple back-connecting API integrates seamlessly into your tech stack, taking the hassle out of proxy management and freeing up valuable coding time. So you can focus on delivering more value from web data.
import requests
response = requests.get(
"example.com",
proxies={
"http": "http://{}:@proxy.zyte.com:8011/".format(APIKEY),
}
)
print(response.text)
?php
use GuzzleHttp\Client;
$client = new Client();
$response = $client->get('http://example.com', [
'proxy' => 'http://APIKEY:@proxy.zyte.com:8011'
]);
echo $response->getBody();
?
require 'curb'
request = Curl::Easy.new("http://example") do |curl|
curl.proxy_url = "proxy.zyte.com:8011"
curl.proxypwd = "APIKEY:"
end
request.perform
puts request.body_str
var request = require('request');
request({
uri: 'http://example.com',
proxy: 'APIKEY:@proxy.zyte.com:8011'
}, function callback(error, response, body) {
console.log(body);
});
import java.io.File;
import javax.net.ssl.SSLContext;
import org.apache.http.HttpHeaders;
import org.apache.http.HttpHost;
import org.apache.http.auth.AuthScope;
import org.apache.http.auth.UsernamePasswordCredentials;
import org.apache.http.client.AuthCache;
import org.apache.http.client.CredentialsProvider;
import org.apache.http.client.config.RequestConfig;
import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.client.protocol.HttpClientContext;
import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
import org.apache.http.conn.ssl.TrustSelfSignedStrategy;
import org.apache.http.impl.auth.BasicScheme;
import org.apache.http.impl.client.BasicAuthCache;
import org.apache.http.impl.client.BasicCredentialsProvider;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.message.BasicHeader;
import org.apache.http.ssl.SSLContexts;
import org.apache.http.util.EntityUtils;
public class ClientProxyAuthentication {
public static void main(String[] args) throws Exception {
// Trust own CA and all self-signed certs
SSLContext sslcontext = SSLContexts.custom()
.loadTrustMaterial(new File("/path/to/jre/lib/security/cacerts"),
"changeit".toCharArray(),
new TrustSelfSignedStrategy())
.build();
// Allow TLSv1.2 protocol only
SSLConnectionSocketFactory sslsf = new SSLConnectionSocketFactory(
sslcontext, new String[] {"TLSv1.2"},
null,
SSLConnectionSocketFactory.getDefaultHostnameVerifier());
CredentialsProvider credsProvider = new BasicCredentialsProvider();
credsProvider.setCredentials(
new AuthScope("proxy.zyte.com", 8011),
new UsernamePasswordCredentials("", ""));
try (CloseableHttpClient httpclient = HttpClients.custom()
.setDefaultCredentialsProvider(credsProvider)
.setSSLSocketFactory(sslsf)
.build()) {
HttpHost target = new HttpHost("example.com", 443, "https");
HttpHost proxy = new HttpHost("proxy.zyte.com", 8011);
AuthCache authCache = new BasicAuthCache();
BasicScheme basicAuth = new BasicScheme();
basicAuth.processChallenge(
new BasicHeader(HttpHeaders.PROXY_AUTHENTICATE,
"Basic realm=\"Zyte\""));
authCache.put(proxy, basicAuth);
HttpClientContext ctx = HttpClientContext.create();
ctx.setAuthCache(authCache);
RequestConfig config = RequestConfig.custom()
.setProxy(proxy)
.build();
HttpGet httpget = new HttpGet("/");
httpget.setConfig(config);
try (CloseableHttpResponse response = httpclient.execute(
target, httpget, ctx)) {
System.out.println(response.getStatusLine());
System.out.println(EntityUtils.toString(response.getEntity()));
EntityUtils.consume(response.getEntity());
}
}
}
}
using System;
using System.IO;
using System.Net;
namespace ProxyRequest
{
class MainClass
{
public static void Main (string[] args)
{
var myProxy = new WebProxy("proxy.zyte.com:8011");
myProxy.Credentials = new NetworkCredential(APIKEY, "");
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("http://example.com");
request.Proxy = myProxy;
request.PreAuthenticate = true;
WebResponse response = request.GetResponse();
Stream dataStream = response.GetResponseStream();
var reader = new StreamReader(dataStream);
Console.WriteLine(reader.ReadToEnd());
reader.Close();
response.Close();
}
}
}
$ curl http://example.com \
-U APIKEY: \
-x proxy.zyte.com:8011
Besides our datacenter proxies, Smart Proxy Manager offers residential proxies. Use the Residential add-on in your active Smart Proxy Manager subscription for geo-blocked websites.
“Zyte was able to offer the most simple and effective rotating proxy solution for us. It just works.”
“It was literally 5 lines of code to get started with Smart Proxy Manager and see crawling success.”
“Without Zyte Smart Proxy Manager our business is not successful.”
Yes, we can allocate IPs based on the locations you choose. You can otherwise leave the automatic IP proxy rotation down to us wholly to us, where our proprietary algorithms will choose the best IPs to get you the successful requests.
It’s in the order of hundreds of thousands. But we have reserves as well depending on the requirements of our customers. We grow the pool as needed.
Yes. We offer a free trial of our smart rotating proxy service for 14 days or 10,000 requests (whichever comes first). Customers interested in an Enterprise Free Trial should contact us.
Yes, you may cancel your subscription at any time
Smart Proxy Manager (formerly Crawlera) is focused on delivering successful responses without any hassle for the end-user. The internal algorithm deals with bans, manages multiple proxy providers, rotates proxies and handles throttling automatically, among other features. It allows your technical team to focus on data only and not proxies.
Definitely. Smart Proxy Manager is a standalone service that can be used with any crawler or HTTP client, independently of the rest of the Zyte platform.
When you start extracting data from the web on a small scale you might not need proxies to make successful requests and get the data. But, as you scale your project because you need to extract more records or more frequently, you will experience issues. Or the site you're trying to reach might display different content depending on the region. So these are the two cases when you need to start using a proxy solution. Learn more about proxies here.
