Web scraping for online cosmetics marketplace

Find out how ScraperDev by extracting data about competitors assortments helped an FMCG company optimize its procurement processes.


Our client is a significant E-commerce beauty and FMCG products marketplace. Due to the huge variety of SKUs on the marketplace, the company met with risks related to assortment planning. They noticed that some products sold out before demand was met, whole others remained unsold for months. The company approached our ScraperDev team looking for a data-driven stable solution to their stocking issues and eliminating assortment inconsistencies.

Industry: Retail and e-commerce

Cooperation: 2021-2022

Location: France

“Reinventing assortment planning solution based on real-time data on the market”

Improved assortment planning system with the possibility to track real-time data on the market


Case Study Tags


Retail industry challenges related to the products’ variety

Old-fashioned ways of doing business have now been overtaken by new, faster, and cheaper methods, such as web data scraping. Web scraping is mainly used by e-commerce businesses to monitor competitors’ prices, reviews, and availability. In this case we will highlight another less obvious application of web data scraping, Product Assortment Planning, and Optimization.  


In order not to overload the competitors' sites we made sufficient intervals between requests and stretched each scraping session over several hours. We were careful to ensure that there was no critical loading put on the competitors' sites which may adversely affect his business.


Building custom scrapers to organize data collection

We created a stable data pipeline with real-time data on the current inventory of the product. Our client could now track all changes in real time, and all the data we collected could be exported for further use. We checked all stocked products and their prices every day. These checks were done in two ways, firstly by writing directly on the site the quantity in the stock. 

Secondly, we wrote a code for algorithm that adds a request to add random numbers of items to the card (for example, 5.000, 30.000, 150.000) to check the “error” which provides data on how many items are specifically on stock now.  



Our client’s assortment process was improved, and led to increased profits of 47% over the following 4 months, with 85% accuracy in demand predicting 


How our services bring about success

Curious how it looks like?

A free data sample will be automatically sent to you.