Data contracts require a change of mindset

I wrote yesterday how using data contracts allows you to:

• Create a stable interface for the data, much like an API
• Automate the running and reporting of data quality checks
• Improve communication between those that create data and those that consume it

But doing that with data contracts isn’t just about building some tools. It requires a change in mindset.

For example, creating a stable interface is only worth doing once you and your stakeholders (in particular, the data consumers) have identified that building on data that doesn’t come through a stable interface, such as when you build on top of someone else’s database either directly or through CDC, is the root cause of many of your pipeline failures.

And then once you’ve identified that, you are your stakeholders agree that using an interface with appropriate change management is worth the added friction, because you all understand the value not of the data pipelines themselves but of the application of data that is impacted and the value that provides the organisation

So now you’re moving towards a mindset where your data producers are willing to put more effort into the production of data, because they understand the value to the organisation of doing so.