Undisciplined use of Kafka leads to poor quality data
·1 min
I was at Confluent’s Data In Motion event in London yesterday and this quote by Tim Berglund in they keynote caught my attention:
Data Contracts
📙 Looking for more on data contracts? Check out my book! 📙
2024
Start with your critical data. Aim for all your data
·1 min
When you start your adoption of data contracts, you want to start with the most critical datasets, because this is where you can add the most value.
Automated schema migrations
·1 min
When a schema change is made in a data contract you might want to think about how you can automate the deployment of that and migration to it, without data loss or effort from the consumers.
Start with the problem
·1 min
What is the problem you’re trying to solve with data contracts?
The data contract captures contextual information
·1 min
The problem with using data isn’t usually finding it - data catalogs can surface that data.
Data contracts cannot assign ownership
·1 min
I was giving a talk a few weeks ago on data contracts, and through it one audience member seemed particularly engaged, nodding along throughout.
A business case that improves data quality
·2 mins
Many data engineering teams spend a lot of their time struggling to deal with upstream data.
You don't need a mandate
·1 min
Following on from yesterday’s note on the API Mandate and data contracts, I wan’t to be clear you don’t need to mandate data contracts to get adoption.
Data contracts and the API mandate
·2 mins
In 2002 Jeff Bezos issued the now infamous API Mandate which stated the following:
Workflows must have a benefit
·1 min
Every workflow and process you have has a cost, including the following:
Want great, practical advice on implementing data mesh, data products and data contracts?
In my weekly newsletter I share with you an original post and links to what's new and cool in the world of data mesh, data products, and data contracts.
I also include a little pun, because why not? 😅
(Don’t worry—I hate spam, too, and I’ll NEVER share your email address with anyone!)