Daily data contracts tips

For a data consumer to know if the data is applicable to their use case, they need to know what to expect from the data.

I wrote yesterday how data quality shouldn’t be the focus.

But even with that, at some point you’ll be looking at a dataset and thinking “is this data good enough for what I want to use it for?”.

Data quality shouldn’t be the focus.

It’s too subjective, and on its own doesn’t really mean much at all.

Are the consumers of your data platform active?

Can they pick what they need from the solutions your providing?

I wrote a few weeks ago about the difference between active vs passive data publishing, and how the active publishing of data leads to better outcomes for your organisation.

Think of the processes you have in place. Which ones are reliant on a single person?

“You broke our data, so your PRs now need our signoff.”

This is a common reaction from data teams who are feeling the impact of upstream data changes causing breakages in their pipelines.

What can you do with data you don’t trust?

Let’s take an example.

I’ve been looking at some data about our cloud costs, and that data is suggesting a clear action I can take to reduce our costs. Which sounds great!

All abstractions leak some details of the complexities it is trying to hide.

And that’s ok!

Your data platform shouldn’t presuppose what can be built upon them.

It should provide the capabilities that allow your users to innovate.