Interfaces, control, and data contracts
Hey all, hope you had a great week! In this week’s newsletter I write about control.
There’s also just a couple of links, one on how metrics influence behaviour, and another on how to work as a data team in “wartime” (i.e. when investments are not growing).
Enjoy the weekend :)
Interfaces, control, and data contracts
You can’t use data contracts — or anything — to gain control over something owned by someone else.
It’s the same as any API or interface.
Stripe could change their API tomorrow, causing breaking changes for everyone who has integrated with it. There’s nothing any of those integrators could do to prevent that change from happening, as they have no control over it.
Of course Stripe won’t, because they know stability is important to their consumers. In fact, they invest in a lot of tooling and reviewing to prevent this happening, deliberately or otherwise.
The stability of their API is fully in their control. And they agree to take on this responsibility because it’s important to their consumers.
Similarly, the authors of Pandas could change the dataframe interface and release it as a patch release, which would break lots of things for their users. As someone who uses Pandas I can’t do anything to prevent that change from happening.
There’s an implicit agreement they won’t, and instead would create a new major version and provide a migration path. But again, only they have the control here, which means they have the responsibility to manage changes to their library as expected by their users.
That’s why the data contract must be owned by the data producer, and not a data consumer.
Only they have control over the data, it’s change management, and it’s quality.
And they need to be incentivised to take on this responsibility, because it’s important to their consumers and, ultimately, the business.
Interesting links
Data Science Without Alignment is Just Expensive Maths by Genevieve Hayes
Choose success metrics that aligns with business problems.
Wartime Data Teams by Joe Reis
Many of us are not getting the investments we were. Instead, Joe argues we should be:
- Focused
- Visible
- Resourceful
- Collaborative
- Agile
Being punny 😅
When I started my first job at a bank, I decided to take notes during my training. Or as my manager called it, “stealing”.
Upcoming workshops
- Implementing a Data Mesh with Data Contracts - Antwerp, Belgium - June 5
- Alongside the inaugural Data Mesh Live conference, where I’ll also be speaking.
- Sign up here
Thanks! If you’d like to support my work…
Thanks for reading this weeks newsletter — always appreciated!
If you’d like to support my work consider buying my book, Driving Data Quality with Data Contracts, or if you have it already please leave a review on Amazon.
Speak soon, Andrew