Terraform

Managing the governance of Cloud resources and configurations in modern data architectures represents a challenge not to be underestimated to avoid unexpected costs. This is crucial when adopting operational models that embrace modern Data Management practices such as decentralizing ownership within data product teams. Automating resource provisioning and establishing architectural standards are essential to enable data product development groups to effectively manage the deployment of their products.

Contatti di riferimento

Giandomenico Avelluto

Strategy Advisor

Overview

Terraform is an open-source Infrastructure as Code (IaC) tool that allows defining, configuring, and provisioning cloud resources using a declarative configuration language.

With Terraform providers and modules, it’s possible to effectively and efficiently manage the lifecycle of infrastructure components by integrating into release pipelines.

Capabilities

Infrastructure as Code (IaC)

Allows you to describe infrastructure resources through definition files, rather than through manual or physical interfaces, in order to automate, speed up and deterministically (or consistently) replicate infrastructure provisioning.

Declarative language

Terraform uses its own declarative language (HCL) to describe infrastructure resources; alternatively other common programming languages such as Python, Java, TypeScript are also supported.

Infrastructure state

Terraform uses a state file in which it keeps track of all the resources that make up the infrastructure at a given time. The state file includes the resources of each resource and the dependencies that exist between them.

Modules

Terraform allows you to structure and organize the source code in a modular way with the aim of making it reusable and maintainable. A module essentially consists of a set of infrastructure resources logically linked together by a common purpose or use.

Platform Thinking

In a modern distributed data architecture, the platform must support data product teams by enabling them to be as autonomous as possible in provisioning the necessary infrastructure components for executing their Data Products, offering services preferably in a self-service mode. For example, they should be able to easily provision topics within a Kafka broker rather than creating the necessary tables within a cloud data warehouse (DWH).

At Quantyca, we have developed a series of architectural blueprints composed of various infrastructure modules (Terraform modules) and applications. These blueprints can be made available to Data Product development teams in self-service mode to facilitate the provisioning and configuration of common components while ensuring the enforcement of the organization’s global policies.