Data Lineage

Every company that uses data to make decisions or carry out operations needs to know the transformations that they undergo throughout their life cycle, from the moment they are created until they are used.

As architectures and organizations become more complex, it is no longer possible to rely on people’s knowledge or documentation. The data lineage allows you to carry out Root Cause Analysis or Impact Analysis activities immediately, improving the efficiency of Data Management projects, optimizing incident response times and minimizing the risk of errors.

Furthermore, sector or generic regulations such as the GDPR recommend correct identification and documentation of data flows as a prerogative for compliance.

An integrated Data Governance solution guarantees the possibility of analyzing data flows, both in terms of dependencies between Data Products, in an approach based on the Data Mesh paradigm, and in terms of relationships between individual columns of the data structures present on the various analytical applications (Column-level lineage).

Using dedicated connectors with Reverse Lineage capabilities it is possible to automatically reconstruct data flows, interpreting the transformations they undergo within the various portions of the Data Platform.

Blindata’s SQL Lineage module uses schema metadata and SQL statements (including standard database objects such as views and routines, query logs, and scripts generated by ELT tools) to infer data flows and transformations.

The automated SQL parser within Blindata generates a SQL syntax tree that vividly illustrates the data flows and transformations present in statements. Next, simplify this representation by eliminating transformations, creating a concise line graph that connects tables and columns only.

To improve user accessibility, the lineage view incorporates drill-down capabilities. This feature allows users to quickly identify the script or routine responsible for generating a specific data stream. Additionally, users can easily drill down into the details of transformations with just a few clicks, facilitating a comprehensive analysis of the underlying processes.

Connectors dedicated to a reporting tool, such as Microsoft PowerBi or Tibco Spotfire, allow automatic end-to-end reconstruction of the lineage, from source to final use of the data.

Specific impact analysis functions highlight, starting from any point of the Data Platform, all the impacted data assets, isolating the sections of interest if necessary.

The integration with Data Quality Monitoring functionality allows you to proactively manage the communication of potential critical issues in the data to all affected parties.