What happens when a data contract is breached?

Hello 👋

This week I write about what happens when a data contract is “breached”, since that is a common question I get asked (although data contracts are not only about enforcement!).

There’s also links to articles on the next data bottleneck, barefoot data platforms, and Clickhouse + Postgres.

Finally, a spicy pun.

What happens when a data contract is breached?

Data contracts are often seen as a way to enforce structure and standards on data.

So, what happens if that data contract is “breached”? What happens when the data doesn’t match the contract?

Firstly, the producer of the data should be the one that is alerted. They should own the incident, and be responsible for resolving it.

Those consuming the data are interested parties in that incident, and should receive regular updates on the resolution.

The aim should be to resolve the impact as quickly as possible whilst limiting its impact.

Once the incident is resolved you should follow a postmortem process to understand what went wrong and what can be done to prevent a similar incident happening in the future.

These “breaches” of the contract shouldn’t be unexpected!

These are complex systems and things will go wrong.

We accept that, assume everyone is working with the best intentions, and learn from our mistakes.

With data contracts, we’re assigning responsibility for these incidents to the data producers, as they are the only ones able to fix the issue at source.

Which is the only place we can improve the quality of the data.

Interesting links

The next data bottleneck by Katie Bauer

Great read. If AI does commoditise data retrieval, then data professionals need to consider where they can really add value.

Barefoot Data Platforms by David Gasquez

A nice exploration into what a truly minimal data platform could look like.

Fast, scalable, enterprise-grade Postgres natively integrated with ClickHouse

Clickhouse + Postgres. That’s quite a combination.

Being punny 😅

I really got into that Netflix series about herbs and spices this weekend. I’m currently on Season 4.

Thanks! If you’d like to support my work…

Thanks for reading this weeks newsletter — always appreciated!

If you’d like to support my work consider buying my book, Driving Data Quality with Data Contracts, or if you have it already please leave a review on Amazon.

🆕 I’ll be running my in-person workshop, Implementing a Data Mesh with Data Contracts, in June in Belgium. It will likely be only in-person workshop this year. Do join us!

Enjoy your weekend.

Andrew