↓Skip to main content

Starting small with data contracts

·3 mins
Data Contracts

Happy Pi Day! Hope you’ve had a good week.

Today I have a small post on starting small, before 3 great reads, and a barely legal pun. Enjoy!

Starting small with data contracts

You don’t need to run a big transformation project to start your adoption of data contracts.

You can start with a small problem you need to solve right now.

For example, let’s say you have a dataset created with dbt, and you want to adopt data contracts to automate the anonymisation of personal data in that dataset and others like it when it exceeds your data retention policies.

You simply:

  1. Create a data contract that categorises the data in the dataset
  2. Ask your data producers to categorise their data
  3. Deploy an anonymisation tool that uses that metadata to anonymise data in that dataset

The dataset itself hasn’t changed. Existing users do not need to do any migration.

But you’ve delivered some value, and started on your journey to data contracts.

Supply Chains Already Solved This — What Metadata Can Learn by Viktor Kessler

Modern businesses don’t just generate data; they transport, transform, and distribute it like goods moving through a supply chain. While data is the shippable product, everything surrounding it — ensuring it reaches the right consumer in the right condition — is metadata.

This is a great analogy, and I fully agree on the importance of metadata.

The Current Data Stack is Too Complex: 70% Data Leaders & Practitioners Agree - by Saurabh Gupta, Animesh Kumar, and Matt Lampe

Interesting stats on how complex data stacks are (or are viewed) and I agree that not only is the outcome poor this complexity leads to high costs — particularly the integration costs.

Cutting Middle Management Is Costing Your Capacity for Change by Katie Leonard (LinkedIn)

Efficiency is not the same as effectiveness. An organization running at 100% capacity is brittle, not agile. The smartest companies will recognize this and ensure they maintain the slack they need to grow.

💯

Being punny 😅

I just moved into a new flat, and directly below me is a police station. It would appear that I am above the law.

Upcoming workshops

  • Implementing a Data Mesh with Data Contracts - Antwerp, Belgium - June 5

Thanks! If you’d like to support my work…

Thanks for reading this weeks newsletter — always appreciated!

If you’d like to support my work consider buying my book, Driving Data Quality with Data Contracts, or if you have it already please leave a review on Amazon.

Enjoy your weekend.

Andrew

↑