Skip to main content

Learnable systems with data contracts

·2 mins

Data contracts provide a standard way for your organisation to create, manage, and consume data.

It does this by first using a standard way to capture the metadata - the data that describes the data.

Once you have that metadata, you can provide common tools and services to manage that data. This could include:

  • Backups
  • Access controls
  • Deletion and anonymisation
  • Data quality checks
  • Data catalogs

Or anything else you need.

A diagram showing tools and services driven by a data contract.

Now, as well as standard metadata, you have a standardised way to create, manage, and consume data.

This means that once someone in your organisation learns how, for example, access management is implemented with data contracts, they know how access management is implemented for all data contracts.

Same goes for each of the other capabilities being provided through data contracts.

The data platform, and your approach to data management, becomes easily learnable.

(Inspired by Robin Rendle’s observation that HTML is learnable by default.)


The Hidden Cost of Over-Abstraction in Data Teams by Zakaria Hajji

This is a good article about abstractions, and how sometimes they add more complexity than they removed.

Good abstractions are hard to build.

See also good abstractions are obvious but difficult to find, even in the cloud by Gregor Hohpe and my post on finding the right level of abstraction.

The Rise of the Declarative Data Stack by Simon Späti

This is a three part blog series on declarative data stacks. I don’t agree with all the specifics, but I do agree with the premise that we should be declaring what we want to happen, not exactly how it happens.

See also my post on a contract-based data platform, which uses data contracts as the declaration format to build a data platform around.

But also, see above on the difficulty of creating good abstractions!


Being punny 😅

I lent my phone to a clumsy detective, and he managed to crack the case.


Upcoming workshops


Thanks! If you’d like to support my work…

Thanks for reading this weeks newsletter - always appreciated!

If you’d like to support my work consider buying my book, Driving Data Quality with Data Contracts, or if you have it already please leave a review on Amazon.

Enjoy your weekend.

Andrew


Want great, practical advice on implementing data mesh, data products and data contracts?

In my weekly newsletter I share with you an original post and links to what's new and cool in the world of data mesh, data products, and data contracts.

I also include a little pun, because why not? 😅

(Don’t worry—I hate spam, too, and I’ll NEVER share your email address with anyone!)


Andrew Jones
Author
Andrew Jones
I build data platforms that reduce risk and drive revenue.