Product data scraping

Automate your product data scraping and save up to 90% of your time compared to custom manual extraction. Action insights quicker with just one click and make informed business decisions.

Introducing product list API tool

Empower your product intelligence initiatives with AI-driven product data extraction at scale to stay on top of competitor research, dynamic pricing, and new product reviews. Get all the data fields you need— from SKUs/GTINs/MPN to stock availability, reviews, and more.

Extracting data from dynamic pages can be a hassle, but our AI algorithms have been trained to fetch the cleanest and most accurate product data without broken spiders or getting banned. Zyte Automatic Extraction supports a comprehensive list of product data scraping fields and with the output delivered directly to your AWS S3 bucket ensuring flexibility to evolve with your needs.

What is API scraping?

API stands for ‘application programming interface’. An API provides a structured way for developers to create programs that can be integrated with other programs or applications.

In the context of web data scraping, you can think of an API as a database interface that’s pretending to be a website. For example, a sports data API might have a URL like some-sports-api.com/getscores?team=some-team - that's like saying "run the get-scores function where the team argument is 'some-team'" so you can get some-team's scores.

API scraping typically returns data in a JSON format. Many APIs will have packages in most major programming languages that will build the requests using standard syntax for the language.

How to scrape product information?

It’s up to you: Take our ready-to-use product data scraping API or use the self-serve interface where you simply enter your URLs in. Either way, Zyte’s product scraper makes it easy to scrape the product information you need.

Is it legal to scrape API?

Using an API to scrape data from public websites is not inherently illegal. However, many APIs require that you sign up and agree to the site’s Terms of Service. These Terms of Service may place legal limitations on how you access, use, and share the data so it is important that you carefully read them before agreeing.

Benefits:

More precise than competitors — read the whitepaper

Access product data at scale, in real-time with unbeatable quality;

Reliable and always-on service; 24/7, 365 days a year;

Easily scrape all the product data fields you need from SKUs/GTINs/MPN to stock availability, reviews, and more;

14-day free trial available

“Saved me so much time; To get all the data I needed using Automatic Extraction it took approx 4 hours across multiple domains. This would have taken a week in the past.”

What data categories will be extracted when scraping products?

• Product name
• Price / Currency
• Availability
• Description
• MPN/GTIN/SKU
• Brand
• Breadcrumbs
• Images
• Rating
• URL
• Pagination
• Reviews

Example output of a product scraper

[
  {
    "product": {
      "name": "Product name",
      "offers": [
        {
          "price": "42",
          "regularPrice": "45.00",
          "currency": "USD",
          "availability": "InStock"
        }
      ],
      "sku": "product sku",
      "mpn": "product mpn",
      "gtin": [
        {
          "type": "ean13",
          "value": "978-3-16-148410-0"
        }
      ],
      "brand": "product brand",
      "breadcrumbs": [
        {
          "name": "Level 1",
          "link": "http://example.com"
        }
      ],
      "mainImage": "http://example.com/image.png",
      "images": [
        "http://example.com/image.png"
      ],
      "description": "product description",
      "aggregateRating": {
        "ratingValue": 4.5,
        "bestRating": 5.0,
        "reviewCount": 31
      },
      "additionalProperty": [
        {
          "name": "property 1",
          "value": "value of property 1"
        }
      ],
      "probability": 0.95,
      "url": "https://example.com/product"
    },
    "webPage": {
      "inLanguages": [
        {"code": "en"},
        {"code": "es"}
      ]
    },
    "query": {
      "id": "1564747029122-9e02a1868d70b7a2",
      "domain": "example.com",
      "userQuery": {
        "pageType": "product",
        "url": "https://example.com/product"
      }
    },
    "algorithmVersion": "20.8.1"
  }
]

Product review output example

[
  {
    "productReviews": {
      "url": "https://example.com/product-review",
      "reviews": [
        {
          "name": "A great tool!",
          "reviewBody": "AutoExtract is a great tool for review extraction",
          "reviewRating": {
            "ratingValue": 5.0,
            "bestRating": 5.0
          },
          "datePublished": "2020-01-30T00:00:00",
          "datePublishedRaw": "Jan 30, 2020",
          "votedHelpful": 12,
          "votedUnhelpful": 1,
          "isVerified": true,
          "probability": 0.95
        },
        {
          "name": "Another review",
          "probability": 0.95
        }
      ]
    },
    "webPage": {
      "inLanguages": [
        {"code": "en"},
        {"code": "es"}
      ]
    },
    "query": {
      "id": "1564747029122-9e02a1868d70b7a3",
      "domain": "example.com",
      "userQuery": {
        "pageType": "productReviews",
        "url": "https://example.com/product-review"
      }
    },
    "algorithmVersion": "20.8.1"
  }
]