Arcese - Velocity

Arcese S.p.A. invests in technologies that enable real time as a paradigm for its data platform in order to create new services and meet the market need for ‘real time visibility’.

Solutions:

Data Platform - Data Products

Technologies:

Confluent - Azure - Debezium

Overview

The Arcese Group is a global logistics operator providing solutions covering the entire supply chain: from land and intermodal transport, sea and air shipping, warehouse management and integrated logistics. Arcese combines knowledge of local markets with decades of experience and expertise in global trade.

Thanks to a direct global presence and collaborations with specialised partners on 5 continents, the Group is the strategic choice for thousands of companies operating in major industrial sectors.

Challenges

→ Velocity: the availability of data as a facilitator in the construction of high value real time services

→ Data as an asset to support decision-making processes and the development of innovative solutions

To stay competitive in the logistics sector, it is more and more necessary to provide its customers with integrated and fast services, possibly in real time, exploiting new technologies to guarantee precise and punctual control of the movement of goods. Digitalisation and the adoption of new technologies are two fundamental pillars in Arcese’s business strategy.

For this reason, we worked first on a specific use case, that one of land or road transport in the LTL sphere, and then on an architectural model that would allow the functionalities and technologies used to be extended more and more widely.

Solution

In concrete terms, as Quantyca we supported Arcese in achieving the following objectives:

Development of an Event Streaming platform for real-time integration of Transport Orders

Leveraging technological tools such as the CDC Debezium and the Confluent Cloud streaming platform, it was possible to process the information of shipments carried out overland in real time mode, significantly reducing the latency required to obtain the tracking events provided by Arcese to its customers.

The Kafka Streams applications created were able to rebuild the transactions that occurred on the source management system in order to generate domain events in a defined standard format.

Data Strategy

In order to achieve the goals of democratising and centralising data, while reducing integration time and costs, a Data Strategy Assessment was performed. An Event Driven architecture was therefore designed, with a polyglot storage strategy based on the Data Fabric concept. In addition, a Data Governance tool was introduced for the definition of a Business Glossary and integration with the various corporate data management processes.

Architecture

Several design and development activities were conducted to achieve the goals set:

Configuration of the Confluent Cloud platform, defining how to connect with the Arcese Cloud
Development and monitoring of the necessary source and sink connectors
Development of Kafka Streams applications using the Sping Boot framework
Development of micro-batching applications using the Spring Boot framework and Spring Batch
Definition and adoption of CI/CD pipelines for code lifecycle management
Definition and implementation of end-to-end Integration Test for all applications involved, to verify the goodness of developments
Implementation of Data Quality applications to perform automated sample tests on the large volume of data produced
Defining scalability and elasticity rules and policies
Adoption of the reproducible architecture paradigm (IaC)

Results

Arcese achieved several goals:

The monitoring of the Transport Order and its execution in the LTL environment went from tens of minutes to a few seconds
The data processed in real time were defined following a Unified Data Model that best reflects business concepts
The introduced architecture can handle high volumes of data, including peak cases, thanks to the extended calculation distribution
Integration costs are significantly reduced, the data produced for sharing is uniform, unambiguous and of certified quality, making it easily extensible and scalable
At all levels of the architecture it is possible to collect metadata that is becoming more necessary for governance processes