Retail pricing analysis

How can a retailer gain a competitive advantage with web crawling?

In today's world, many retailers have a hard time to stay active in their very competitive sector. Constantly changing prices and promos make it tough to follow the market. Gaining insight into the general structure of the market and into competitors' pricing strategies can significantly simplify this process. element61 has built an end-to-end solution from scraping webdata from Belgian retailers, to gathering insights through an interactive PowerBI dashboard.

To access our 12 page long white paper on this subject, download below.

Where does the retailer data come from?

The dashboard contains a mix of mockup data (which you would leverage from your own systems) & competitor web data. This data is acquired through a python script that ran daily to crawl the web data.

To access the web data, we leveraged a Python package called Scrapy which allowed us to retrieve, parse & extract the data from various retailer web pages.

What learnings to take from this demo towards your organization?

This demo is a fully-operational working data platform running with Microsoft Azure and Power BI. We see 3 key learnings as very relevant to share with you

  1. Power BI is great to embed: As you see in above embedded, it's easy and user-friendly to embed Power BI in your own website or Customer Portal.

    Embedding Power BI can simplify and boost the use of reports and data: e.g., operators can use Power BI from within the software they use day-to-day (e.g. Dynamics, CRM, etc.)
  2. Easily leverage existing Python packages: In this demo we use an existing web crawling package - Scrapy - to unlock retailer data easily. Leverage the power of open-source Python packages to help out on various use cases.
  3. Agile to develop: The NMBS data demo platform is built using various Azure existing services jointly enabling this end-to-end working set-up. Without re-building from scratch, we can configure the various Azure building blocks and build this demo platform in less than 30 days

What did we learn at element61?

During this exercise, we learned that if you are a retailer or a brand, there are lots of opportunities with publicly available data. Register for our whitepaper to discover how you could leverage this data into a competitive advantage.

Download the white paper

You must agree before submitting.

Thank you for requesting your free copy of the white paper! You will receive it shortly in your inbox.

Enjoy the read and if you have any questions, get in touch via

Want to know more? Continue reading on an Azure Modern Data Platform Architecture

Continue reading about the element61's Modern Data Platform approach.

« Back to homepage