We’ve made a change. Scrapinghub is now Zyte!
Enterprise grade proxy management logic, without the hassle of maintaining it.
Zyte uses machine learning and heuristics to rotate and select the optimal IP address for the specific target website.
Zyte Smart Proxy Manager provides access to both residential & datacenter proxies.
Datacenter IPs are cost effective yet powerful in the right hands. But sometimes that is not enough and residential IPs are a must.
Gain access to both sets of proxies within a single solution.
Open-data at your fingertips. With +98% avg. success rate across all domains, open-data for all is now a reality.
In-built anti-ban management yet compliant web data extraction. Ethically sourced and maintained IPs.
Zyte’s rotating proxy network is built with a proprietary ban detection and request throttling algorithm and hundreds of heuristics, ensuring we achieve the highest and most consistent success rates in the market.
Built to scale with your project. From 100K to billions of requests per month, we help all business sizes collect mission-critical data.
Ethically sourced IPs through a range of partners that undergo stringent due diligence processes to provide you peace of mind & reassurance.
Zyte also performs KYC due dilligence processes on all our residential proxy customers to ensure no nefarious use case or activity goes leverages our network.
Built by developers, for developers. Zyte’s rotating proxy is used today by our 100+ web scraping developers to deliver accurate and reliable data to our Fortune 500 Managed Data customers with stringent SLAs.
Zyte Proxy Management is built specifically for web scraping developers and is supported by a team of web scraping professionals that speak your language and understand your pain points.
Zyte Smart Proxy Manager is probably the best rotating proxy service for scraping. It is designed as a smart downloader, where your spiders just have to make a request to its API and the smart rotating proxy server will return the data you require. Managing all the proxy rotation, throttling, blacklists, session management, etc. under the hood so you don’t have to.
import requests
response = requests.get(
"example.com",
proxies={
"http": "http://{}:@proxy.zyte.com:8011/".format(APIKEY),
}
)
print(response.text)
?php
use GuzzleHttp\Client;
$client = new Client();
$response = $client->get('http://example.com', [
'proxy' => 'http://APIKEY:@proxy.zyte.com:8011'
]);
echo $response->getBody();
?
require 'curb'
request = Curl::Easy.new("http://example") do |curl|
curl.proxy_url = "proxy.zyte.com:8011"
curl.proxypwd = "APIKEY:"
end
request.perform
puts request.body_str
var request = require('request');
request({
uri: 'http://example.com',
proxy: 'APIKEY:@proxy.zyte.com:8011'
}, function callback(error, response, body) {
console.log(body);
});
import java.io.File;
import javax.net.ssl.SSLContext;
import org.apache.http.HttpHeaders;
import org.apache.http.HttpHost;
import org.apache.http.auth.AuthScope;
import org.apache.http.auth.UsernamePasswordCredentials;
import org.apache.http.client.AuthCache;
import org.apache.http.client.CredentialsProvider;
import org.apache.http.client.config.RequestConfig;
import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.client.protocol.HttpClientContext;
import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
import org.apache.http.conn.ssl.TrustSelfSignedStrategy;
import org.apache.http.impl.auth.BasicScheme;
import org.apache.http.impl.client.BasicAuthCache;
import org.apache.http.impl.client.BasicCredentialsProvider;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.message.BasicHeader;
import org.apache.http.ssl.SSLContexts;
import org.apache.http.util.EntityUtils;
public class ClientProxyAuthentication {
public static void main(String[] args) throws Exception {
// Trust own CA and all self-signed certs
SSLContext sslcontext = SSLContexts.custom()
.loadTrustMaterial(new File("/path/to/jre/lib/security/cacerts"),
"changeit".toCharArray(),
new TrustSelfSignedStrategy())
.build();
// Allow TLSv1.2 protocol only
SSLConnectionSocketFactory sslsf = new SSLConnectionSocketFactory(
sslcontext, new String[] {"TLSv1.2"},
null,
SSLConnectionSocketFactory.getDefaultHostnameVerifier());
CredentialsProvider credsProvider = new BasicCredentialsProvider();
credsProvider.setCredentials(
new AuthScope("proxy.zyte.com", 8011),
new UsernamePasswordCredentials("", ""));
try (CloseableHttpClient httpclient = HttpClients.custom()
.setDefaultCredentialsProvider(credsProvider)
.setSSLSocketFactory(sslsf)
.build()) {
HttpHost target = new HttpHost("example.com", 443, "https");
HttpHost proxy = new HttpHost("proxy.zyte.com", 8011);
AuthCache authCache = new BasicAuthCache();
BasicScheme basicAuth = new BasicScheme();
basicAuth.processChallenge(
new BasicHeader(HttpHeaders.PROXY_AUTHENTICATE,
"Basic realm=\"Zyte\""));
authCache.put(proxy, basicAuth);
HttpClientContext ctx = HttpClientContext.create();
ctx.setAuthCache(authCache);
RequestConfig config = RequestConfig.custom()
.setProxy(proxy)
.build();
HttpGet httpget = new HttpGet("/");
httpget.setConfig(config);
try (CloseableHttpResponse response = httpclient.execute(
target, httpget, ctx)) {
System.out.println(response.getStatusLine());
System.out.println(EntityUtils.toString(response.getEntity()));
EntityUtils.consume(response.getEntity());
}
}
}
}
using System;
using System.IO;
using System.Net;
namespace ProxyRequest
{
class MainClass
{
public static void Main (string[] args)
{
var myProxy = new WebProxy("proxy.zyte.com:8011");
myProxy.Credentials = new NetworkCredential(APIKEY, "");
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("http://example.com");
request.Proxy = myProxy;
request.PreAuthenticate = true;
WebResponse response = request.GetResponse();
Stream dataStream = response.GetResponseStream();
var reader = new StreamReader(dataStream);
Console.WriteLine(reader.ReadToEnd());
reader.Close();
response.Close();
}
}
}
$ curl http://example.com \
-U APIKEY: \
-x proxy.zyte.com:8011
Smart Proxy Manager gives developers the flexibility to turn their ideas into reality and the support to scale them into production.
With a simple back connecting API that integrates seamlessly into your workflow, we take the hassle of proxy management and free up valuable time. So you can focus on driving value out of web data.
