Daily data contracts tips

Data products are a term that is increasingly being talked about, but one which seems to have different meanings to different people.

On the latest episode of the Data & AI podcast I joined Deepak and Dan as we demystified data contracts.

When building a data platform I’m often thinking about how to reduce friction for my users. I want them to have a great experience that allows them to get/build what they need quickly and easily.

Data engineers typically favour moving quickly over reliability.

That’s not how most of us would say it, but it’s true!

On Thursday I’ll be presenting at the AIDA User Groups Pi Day event.

I’ll be talking about data quality, and why prevention is better than the cure.

Many data platforms start with a change data capture (CDC) service to extract data from an organisations transactional databases - the source of truth for their most valuable data.

There is strong alignment between the goals of data mesh and data contracts.

In fact, data mesh was one of the inspirations behind data contracts.

If you’re creating data, you’re modelling data.

You’re making a decision on how that data is presented to your users.

Computation is the most expensive part of your data stack.

So, if you’re thinking about becoming more cost effective, you need to reduce the amount of computation you need to do in order to produce value for the business.

I enjoyed this post by Ryan Collingwood on what the 🤬 is a Data Contract? (which he also published on LinkedIn). It’s a great summary of some of the qualities he feels define a data contract.