We’ve made a change. Scrapinghub is now Zyte! 

Forum data extraction

API Available
Beta

Extract structured forum data at scale in just a click

Get access to clean and relevant forum data at scale, in real-time with optimal quality. Focus on analyzing, not on data preparation and harmonization. Through the use of deep learning, computer vision, and our proxy network, forum data is extracted from forums without the need for you to develop and maintain code.

With an AI-enabled automatic extraction API, that is resilient to website/webpage changes, forum data is turned into insight ready schemas applicable to use cases such as sentiment analysis, Voice of Customer (VoC), Voice of Employee (VoE) and more.

Zyte Automatic Extraction supports a comprehensive list of metadata types and with the output delivered directly to your AWS S3 bucket ensuring flexibility to evolve with your needs.

Data specification

  • Topic
  • Text
  • Date Published
  • Upvote count
  • Reply count

Output example

[
  {
    "forumPosts": {
      "url": "https://example.com/forum-topic-1",
      "topic": {
        "name": "Which is the best country to work in?"
      },
      "posts": [
        {
          "text": "Finland is often considered the best for it.",
          "datePublished": "2020-01-30T00:00:00",
          "datePublishedRaw": "Jan 30, 2020",
          "upvoteCount": 12,
          "replyCount": 1,
          "probability": 0.95
        },
        {
          "text": "Switzerland has good work life balance.",
          "upvoteCount": 2,
          "probability": 0.80
        },
        {
          "text": "Depends on the person",
          "replyCount": 1,
          "probability": 0.80
        }
      ]
    },
    "webPage": {
      "inLanguages": [
        {"code": "en"},
        {"code": "es"}
      ]
    },
    "query": {
      "id": "1564747029122-9e02a1868d70b7a3",
      "domain": "example.com",
      "userQuery": {
        "pageType": "forumPosts",
        "url": "https://example.com/forum-topic-1"
      }
    },
    "algorithmVersion": "20.8.1"
  }
]

Get sample data

Ready to get started?

Your Data Extraction Partner

Our web scraping team of experts can provide services to suit any size of business, from twinkly startups to Fortune 100’s.

Talk to us

Zyte Automatic Extraction

Our AI enabled always-on, on-demand web data extraction solution.

Try free