Selex - Retail Analytics in Cloud

Retail & FMCG

Selex builds new retail analytics platform in Cloud with Snowflake

Solutions:

Data Platform - Data Infrastructure

Technologies:

Aws - Talend

Quantyca Retail Analytics in Cloud Selex

Business Summary

Selex Gruppo Commerciale is the second largest Italian operator in modern organized distribution with a market share of over 14%. Its ability to intercept market changes and understand consumer needs has enabled the Group of 18 members to develop in a capillary manner throughout the Peninsula.
Today, the commercial network consists of more than 3,200 points of sale integrated in every territory and available in all formats – from cash and carry to hypermarkets, supermarkets and superettes.

The drive to constantly improve its offering led Selex to launch the CosìComodo online shopping portal in 2017. Since then, the e-commerce channel has grown steadily over the years and now includes 12 of the Group’s brands, for a total of over 100 pick-up points and 400 CAP territories covered by the home delivery service.
To guarantee consumers increasingly innovative, efficient and top-quality products and services Selex decided in 2020 to equip itself with a retail analytics platform, selecting Snowflake as the basis for managing data from the digital channel and PowerBI for generating the analyses.
The new platform, developed in collaboration with Quantyca over a period of 6 months, has enabled Selex to have a constantly updated and integrated view of all the key indicators along the main dimensions of analysis (customer, product, promotion, member, brand, …), thus reducing the time and cost of producing new reports and enabling a data-driven approach to the definition of new business initiatives.

Challenges

The analysis of sales and customer behavior is central for Selex to support all kinds of decisions and to constantly provide an innovative and quality service. The constant growth in the number of touchpoints and sales channels, however, makes it increasingly complex for retailers to consolidate the large amount of data collected in order to produce consistent and up-to-date views of the main business dimensions on which they can then reason and make decisions.

This complexity is even greater for Selex because, as a trading group composed of several independent partners, it has to manage greater fragmentation and heterogeneity of the data during the acquisition phase and more stringent security and profiling constraints during distribution and analysis.

We knew that in order to be able to continue offering our customers a high-quality service, we had to better exploit the potential contained in the data collected by CosìComodo to support us in our decision-making processes
Andrea Nobili - IT eCommerce & Digital Manager at Selex

The pre-existing BI solution based on a fixed set of standard reports implemented by means of a custom product directly on top of the operational feeder systems no longer guaranteed the desired scalability and flexibility as volumes and consequently analytical needs grew.

The adoption of PowerBI as a new reporting product simplified data collection and made reports more flexible and interactive. However, the costs and time required to implement new analyses remained substantially unchanged due to the necessary integrations on the upstream data, reproduced for each report with ad-hoc logics, lacking a platform capable of centralizing and rationalizing this work. Moreover, many of the transformation logics being implemented directly within the reporting tool reduced its performance, negatively impacting the user experience.

Solution

Selex selected Snowflake as the solution behind the new retail analytics platform in order to increase the scope of analysis while reducing costs and development time. Thanks to Snowflake and the support of Quantyca, it was possible to realize in 6 months of the project a platform able to rationalize integrations and produce an integrated view of the data of interest that can be used both for the realization of institutional reporting and to quickly generate ad-hoc reports or provide the end user with self-service BI functionalities.

Thanks to Quantyca, we have released a new retail analytics platform based on Snowflake, which has allowed us to increase the scope of analysis related to the digital channel while reducing costs and development time
Andrea Nobili - IT eCommerce & Digital Manager at Selex

Architecture

At the architectural level, the core of the implemented solution is a multi-level Data Warehouse in which data is collected from the source systems within the staging level, then consolidated and historicised in the business level to be finally aggregated and exposed to analysis tools in the application level.
The modeling of data within the business layer is designed to facilitate analysis and facilitate the reuse of data in multiple reports and use cases.

At this level, the decoupling of data consumers and data producers is ensured in order to rationalize integrations and progressively reduce the development time of new analyses. At the application level, on the other hand, data is aggregated and modeled in structures called datamarts designed to optimize the performance of analyses linked to specific business domains (e.g. sales datamart, promotions datamart, etc..). Thanks to datamarts, it is possible to nimbly navigate within a data domain by adding or removing analysis dimensions (slice & dice) and changing the aggregation levels of the measures of interest along the desired dimensions (roll up & drill down).

Integrations

The integrations developed with Talend manage data acquisition from source systems and consolidation within the DWH. Acquisition takes place using incremental logic. At each iteration, only new or changed data since the last upload is loaded, thus reducing computation time and costs. It is also possible to perform the uploads in high-frequency schedulable micro-batch logic so that the DWH always has up-to-date data.

Once the delta data has been loaded in the staging layer, subsequent transformations take place in ELT logic, i.e. processing the data directly in the database. This makes it possible to make full use of Snowflake’s computational and scaling capabilities by avoiding further data shifts.

Infrastructure

The infrastructure was built entirely in the Cloud on AWS and is divided into three environments: development, quality and production. The management of the environments in terms of both security and synchronization is delegated to Snowflake.

Snowflake

The choice of Snowflake made it possible to realize a highly scalable, secure and operationally easy-to-manage cloud solution in a short time. Below are some of the features of Snowflake that brought the main benefits at the design level.

High performance and low cost through separation of storage and computation

Snowflake’s cloud-native architecture based on the separation of storage and computation made it possible to allocate resources optimally by creating virtual data warehouses on top of the storage specifically dimensioned for different types of workloads (integration, analysis, etc.). The virtual warehouses are started at the start of the associated workload and then switched off, allowing significant savings in computation costs as all workloads in the field, including the analysis workload, are in an idle state for significant percentages of the operating time.

Environments always synchronized thanks to zero copy cloning

The three environments on which the proposed solution is built are always up-to-date with each other thanks to the zero copy cloning functionality that allows data to be replicated between environments without making physical copies of the data itself. Thanks to this functionality, it is therefore not necessary to develop ad-hoc integrations to keep the test and quality environments aligned with production. Moreover, since no physical copies of the data are made, update times between environments are very fast and do not entail extra storage costs.

Data security and segregation thanks to profiling and sharing functionalities

The granular data profiling functionalities offered by Snowflake made it possible to meet the stringent security and segregation requirements of a multi-company company such as Selex without impacting development costs and agile information sharing needs.

Results

Thanks to the new retail analytics platform, Selex has reduced the time and cost of carrying out new analyses, thus responding more effectively and efficiently to business needs.
The possibility of analyzing integrated and constantly updated views from the data coming from the different sources allows Selex to follow a data-driven approach to the evolution of its commercial channels in order to provide its customers with a constantly high quality and innovative offer.

Finally, the implemented solution allows Selex to easily manage the acquisition, securing and sharing of data in a multi-company reality characterized by different IT and analysis requirements.

Risorse

Whitepaper

Free

17/06/2022

Selex – Retail Analytics in Cloud con Snowflake