Skip to main content

Are data contracts only for tabular data?

·1 min

It’s a misconception that data contracts are just for tabular data.

Sure, they work really well for tabular data in a data warehouse, and that’s where many people start their implementation.

But we make extensive use of data contracts for our streaming pipelines too, supporting the event-driven architecture of our software engineering teams.

And that’s just as easy to do.

Taking BigQuery as a tabular example, you convert your data contract to the custom JSON format BigQuery understands and use that to create the table.

Flowchart showing a data contract being converted into a custom JSON file, which is then analyzed with a magnifying glass icon displaying a bar chart.

With Kafka as a streaming example, you convert your data contract to protocol buffers (protobuf) and apply that schema to a topic.

A diagram showing a data contract document with an arrow pointing to a protocol buffers document, which then connects to a network diagram with a central node and six surrounding nodes.

Both of these interfaces are now under the change management of your data contract and the rules you enforce through your data contract tooling.

So, data contracts are just as applicable to streaming data as they are to tabular data.

In fact, data contracts are applicable to any form of structured data where you need change management on that structure.


Want great, practical advice on implementing data mesh, data products and data contracts?

In my weekly newsletter I share with you an original post and links to what's new and cool in the world of data mesh, data products, and data contracts.

I also include a little pun, because why not? 😅

(Don’t worry—I hate spam, too, and I’ll NEVER share your email address with anyone!)


Andrew Jones
Author
Andrew Jones
I build data platforms that reduce risk and drive revenue.