As you know we held the first-ever Web Data Extraction Summit last month.

In this article, I will guide you through a web scraping and data visualization project.

The Web Data Extraction Summit was held last week, on 17th September, in Dublin, Ireland.

A huge portion of the internet is news. It’s a very important type of content because there are always things happening either in our local area or globally that we want to know about.

Product data—whether from e-commerce sites, auto listings, or product reviews—offers a treasure trove of insights that can give your business an immense competitive edge in your market

The manual method of discovery for gauging online public sentiment towards a product, company, or industry is cursory at best, and at worst, may harm your business by providing incorrect or misleading insights.

The range of use cases for web data extraction is rapidly increasing and with it the necessary investment. Plus the number of websites continues to grow rapidly and is expected to exceed 2 billion by 2020.

Configuring and using proxies is not easy, especially when sending HTTP requests.

One common misconception about scraping personal data is that public personal data does not fall under the GDPR.

In the fifth and final post of this solution architecture series, we will share with you how we architect a web scraping solution, all the core components of a well-optimized solution, and the resources required to execute it.

In the fourth post of this solution architecture series, we will share with you our step-by-step process for evaluating the technical feasibility of a web scraping project.