Enforce standardisation with data contracts

Data is published by many different systems, and as the number of systems increase it’s likely your organisations data will become less standardised over time, with different identifiers, different semantics, and so on.

This then makes it more difficult, and more costly, to join the datasets together, making it harder to use and build upon.

One way to solve this problem is to try to define a standard for data producers to follow.

Going further, code libraries and other platform features could be provided to make it easier to follow the standard and/or to enforce it. For example, that library could have a method called create_customer_event(...) and that will ensure the customer ID is valid at the time the event is created.

I’ve heard of a couple of data platform teams looking at doing this with data contracts. I’m interested to see if more organisations try this and the success/learnings they get from it.