Daily data contracts tips

In 2002 Jeff Bezos issued the now infamous API Mandate which stated the following:

1. All teams will henceforth expose their data and functionality through service interfaces.
2. Teams must communicate with each other through these interfaces.
3. There will be no other form of interprocess communication allowed: no direct linking, no direct reads of another team’s data store, no shared-memory model, no back-doors whatsoever. The only communication allowed is via service interface calls over the network.
4. It doesn’t matter what technology they use. HTTP, Corba, Pubsub, custom protocols — doesn’t matter.
5. All service interfaces, without exception, must be designed from the ground up to be externalizable. That is to say, the team must plan and design to be able to expose the interface to developers in the outside world. No exceptions.
6. Anyone who doesn’t do this will be fired.
7. Thank you; have a nice day!

Now, at the time creating APIs were not as easy as they are now, so building these interfaces would take time.

Every workflow and process you have has a cost, including the following:

• Managing your infrastructure as code as part of your development workflow
• Creating project and solution design documents and getting wide feedback as part of your project workflow
• Running automated tests with CI/CD as part of your deployment workflow

Each of these slow down delivery:

I wrote yesterday how it’s best to validate as early as possible.

But there are times that might not be possible, maybe because:

It’s always best to do your data validation as early as possible as that:

• Allows them to be acted on quicker
• Alerts the team with the most context of the data/issue
• Reduces the potential of having the incorrect data in multiple systems

So with data contracts I encourage data producers to validate their data before they publish their data to a downstream service.

As a software engineer, we would never make our database directly available to consumers.

Instead, we would always create an API.

The 98/2 rule as described in the Farnam Street newsletter back in July has really stuck with me:

I see data contracts as being at the cross-section of best practices.

In particular, they take the best practices from APIs, Data Governance, and Platform Engineering.

I mentioned yesterday how by using friction we can influence how users behave and how the business is organised. That’s because friction is a great lever

Yesterday I wrote about Conway’s law and the challenges it causes in organisations where data teams are placed in a separate part of the organisation to where the data is generated.

Conway’s law asserts that the designs of systems are influenced by an organisation’s communication structure, suggesting that how teams are organised and communicate will reflect in the final product.