Daily data contracts tips

Let’s say 20 years ago you ran your code in an environment you configured simply as python. The obvious default would have been Python 2. But today the obvious default is Python 3. If you deployed that same code with the same configuration, what Python environment would you expect? What would you expect that to be in 20 years time?

How do you like your data?

Do you want it to be agile? So it can change at any moment, depending on the needs or wants of those producing the data? If a team decides it wants to model an object differently, with different IDs, they can do so. They are moving fast and breaking things.

It’s that time of year again where teams everywhere are considering a code freeze for the holiday season. Should we have one? How long for? What will we do while our code is frozen? Will we have lots to merge in January, and how will we manage that?

Data governance is changing.

With the move towards data mesh we’re seeing a move away from centralised governance teams defining policies and assigning roles like “data steward” (which no one wanted to be!) to people in the organisation and instead delegating those responsibilities to those teams and domains who produce the data and have the most context on the data.

While yesterday I wrote about prioritising data quality projects, I think it’s important to note that while for many of us the quality does need to be improved, we don’t need to be aiming for perfect.

It’s likely going to be difficult to get a project prioritised if the goal is defined only as “improving data quality”. So what? What’s the business value that will provide or enable? And does that align with the organisations wider goals?

After mentioning the title of my talk for apidays in Paris was “Data Contracts: The API for Data”, I had a couple of responses saying that data contracts are not APIs, and that they’re for communication and facilitating an agreement between the generators and consumers of data (or something along those lines).

If you’re building key product features on top of your data, shouldn’t it be as dependable as your software-backed features?

I’m writing this on the train back from Paris, having spent the last few days at the apidays conference, where I gave a talk titled “Data contracts: The API for Data”. I have a lot of digest and plenty of notes to go through! But one think that struck me is the similarities between software and data.

Interfaces are powerful. That’s why we see them everywhere in software engineering. In fact, I’d say they are essential when you want to depend on something provided by someone else.