Increase collaboration with interfaces

Welcome to my new and improved newsletter! I’ll be publishing in this new format weekly on Friday mornings, discussing how to implement data mesh, data products and data contracts. I hope you enjoy it, but as always you can unsubscribe at any time.

Jeff Bezos’ API mandate is well known, but as a reminder in 2002-ish Bezos sent the following internal memo:

All teams will henceforth expose their data and functionality through service interfaces.
Teams must communicate with each other through these interfaces.
There will be no other form of interprocess communication allowed: no direct linking, no direct reads of another team’s data store, no shared-memory model, no back-doors whatsoever. The only communication allowed is via service interface calls over the network.
It doesn’t matter what technology they use. HTTP, Corba, Pubsub, custom protocols — doesn’t matter.
All service interfaces, without exception, must be designed from the ground up to be externalizable. That is to say, the team must plan and design to be able to expose the interface to developers in the outside world. No exceptions.
Anyone who doesn’t do this will be fired.

But what was the problem Jeff and Amazon were trying to solve by mandating APIs?

The problem was cross-team communication.

Jeff realised that their delivery speed was slowing, because as Amazon grew it took more and more people and teams talking to each other to get anything done.

That took time, created a strong coupling between those teams, and because it was all done person-to-person it was never a stable, reliable, well documented interface, so things kept breaking.

That’s why Jeff wanted everything to go through an APIs. APIs are strictly documented interfaces that define what you can send to them, and what you can get back, and how it will perform (SLOs).

In data, we have the exact same problem.

Our ability to deliver value to our customers is hamstrung because we constantly have to talk to data producers to work out what the data is, where does it come from, how often it’s updated, how can it be joined with this other data, and so on.

Then we have to keep all of that in mind while we use the data.

And then, it changes, without warning.

What we thought we knew about the data changed, our downstream applications broke, and we now have to spend time fixing it - which starts by asking all those questions again.

This repeats, over and over again, wasting hours and hours of time that should be spent delivering value.

The problem is the same, and so is the solution.

We need to use interfaces between data producers and data consumers.

That interface is a data contract.

How to streamline Data Product Management with Data Contracts

On December 4th at 11AM CET I’ll be speaking for 10 mins (pre-recorded) on how data products and data contracts revolutionise data management.

That will be followed by Paolo Platter discussing automation and data product development with Witboost.

Web links of the week

Being punny 😅

When one door closes, another door opens. Other than that, it’s a pretty good car.

Upcoming workshops

Implementing a Data Mesh with Data Contracts - Antwerp, Belgium - June 5 2025
- Alongside the inaugural Data Mesh Live conference
- Sign up here

Thanks! If you’d like to support my work…

Thank you for reading this weeks newsletter - always appreciated!

If you’d like to support my work consider buying my book, Driving Data Quality with Data Contracts, or if you have it already please leave a review on Amazon.

I’d also love to hear what you think of this refreshed newsletter, particularly if you’re happy for me to publish a testimonial on my website. Just hit reply!

Enjoy your weekend, Andrew