Data Contracts

It’s a misconception that data contracts are just for tabular data.

Sure, they work really well for tabular data in a data warehouse, and that’s where many people start their implementation.

What’s the right organisational structure for a successful data contracts implementation?

The reality is every organisational structure is imperfect, and you’re unlikely to be in one that explicitly aligns you with your data producers.

A question I received on a LinkedIn post about change management with data contracts asked (lightly edited for clarity):

Data governance initiatives often deliver a set of rules for people to follow when they create and manage data.

When it comes to data contracts I’m always talking about well structured data.

That’s deliberate, as we use data contracts to make data available to others (teams/groups/domains), and the aim is to provide data that is easy to consume with confidence.

I was asked on LinkedIn how I might approach data contracts with dbt.

I have thought a little about how to apply data contracts to dbt targets, but not a lot and I haven’t done this in practice yet.

Many organisations have a data catalog as a part of their data stack. But as we move towards a more decentralised model with data mesh and data contracts it’s worth revisiting who the data catalog is for.

The whole point of data mesh is to make data in an organisation more accessible.

To build reliable data products we need a data platform that provides us capabilities to do so.

I wrote yesterday about how important a data catalog becomes when you move to a decentralised and autonomous data architecture built on data contracts.