Daily data contracts tips

Data contracts are typically used to apply change management to data in tables, in a data warehouse.

Implementing data contracts at the platform layer gives it the capabilities needed to enable domain ownership.

Software engineers often have many capabilities at the platform layer that helps them deploy changes with confidence, including:

When:

• One team provides infrastructure for another
• One team provides code libraries for another
• One team provides an API for another
• One team provides data for another

Each of those are a product being delivered by one team to the next, and that product must meet the consumers requirements.

Data Contracts are about setting expectations, including:

• How the data is structured, and how that can or cannot change
• Who owns the data, and the support they are providing
• When the data is updated, and how complete it is

And so on.

Software engineers use processes and tooling to instil discipline into the creation of software.

They:

There are 2 parts to implementing data contracts.

One is the tooling that allows your data producers to provide data through an interface. That’s where you implement change management and other data governance policies.

A data contracts implementation doesn’t have to be complicated.

There’s no reason you couldn’t create a minimal implementation in an hour.

dbt Lab’s 2024 State of Analytics Engineering found 57% of data practitioners highlight data quality as one of their chief obstacles in preparing data for analysis.

It’s all very well saying, as I did yesterday, that data producers need to do more to provide us better quality data the data we need to solve our problems.