Price Intelligence is leveraging web data to make better pricing, marketing, and business decisions. Basically, it is all about making use of the available data to optimize your pricing strategy, making it more competitive, increasing profitability, and ultimately, improving your business performance.
From competitor monitoring to dynamic pricing and MAP monitoring, web extracted pricing data has endless uses. Brands and e-commerce companies use pricing data to monitor an overall view of the market. Dynamic pricing can be used to make automatic pricing decisions based on competitor’s data combined with internal data so that you always remain profitable. MAP or Minimum Advertising Price monitoring is a technique that uses web extracted data to ensure the resellers and partners are maintaining the pricing according to the company policies.
During our webinar on “Fuelling Price Intelligence With Web Data Extraction” in June 2020, we got a lot of questions related to the processes and challenges of pricing data extraction. We cover a few important questions here:
A: It varies from website to website, but the general idea is to find the pages where such promotion codes are available and build the logic of looking up code and applying it (clicking a button or sending an AJAX request) into your extraction code.
A: Websites showcase erroneous pricing data when they detect you scraping regularly. This especially happens when you are trying to scale - i.e scrape a lot of products very frequently. Erroneous pricing is not easily recognizable, but comparing the prices or other data fields with previously extracted data and manually checking if there is a big difference in the extracted data can help.
The long-term solution for this would be to be smarter about how you scale and be more thoughtful about the proxy solutions you use.
A: Scraping accurate data is all about having a reliable quality assurance process. The first step towards this process is to have a well-defined JSON schema. Your QA process needs to be a balanced combination of automated ways of testing the data as well as manual ways. This blog post gives a detailed description of data validation techniques.
A: There are many ways to conduct product matching. The main idea would be to gather as many product-specific parameters as you can about the product that you want to match and then compare those parameters.
Eg: For a TV, the product-specific parameters would be resolution, weight, sound, etc. If in comparison, 90% of the parameters of any two products are the same, there is a high chance that it is the same product across two websites.
You can build models on this concept to identify product duplication.
Want to know more about how you can fuel your price intelligence decisions with web extracted data? Watch this webinar where our Technology Evangelist, Attila Toth, takes you on a deep dive through the main challenges affecting price intelligence projects from both a business and technical perspective, and more importantly, how you can solve them.
If you have any more questions or queries on Price Intelligence data extraction, feel free to leave a comment below and we will try our best to answer them.