Data contracts and the meta grid

This weeks post is all about metadata, and how data contracts feeds into the meta grid.

There’s also links to the importance of metadata to GenAI, SQL games, and thinking of DuckDB as a protocol.

Data contracts and the meta grid

Over the last few years people have slowly realised the importance and the power of metadata, i.e. the data that describes the data.

Someone who has been particularly vocal on this trend is Ole Olesen-Bagneux, author of the upcoming book Fundamentals of Metadata Management, who defines metadata as follows:

Metadata is a description that is both attached with what is described and placed somewhere else, to make what is described discoverable and manageable.

I like this definition, and it aligns nicely with how I think of data contracts.

A data contract is simply a collection of metadata that describes the data. A simple idea, but because of the power of metadata, it itself becomes powerful.

The data contract is attached to the data, in that it is defined alongside the code that generates the data and therefore will stay updated and in sync with the data as the data itself evolves.

But the metadata in the data contract is also placed somewhere else to drive something else.

For example we might expose this metadata in a data catalog so users can easily find and access data across the organisation.

And we might also place this metadata in tooling to implement data platform capabilities such as data retention and access management.

And so on.

We can place it wherever we need to to deliver some capability or other value to the producers and/or the consumers of the data.

Importantly, the source of truth for this metadata remains the data contract, but it’s value is realised by having that metadata made available elsewhere.

Ole has gone further and coined the term meta grid as he considers how the power of other metadata, often imperfect, incomplete, and unstandardised across an organisation can be unlocked.

Here, the data contract becomes one source of metadata that can be combined with others to drive processes or other value.

One example I can think of would be to combine the metadata from a data contract with metadata on your data governance policies to create a Record of Processing Activities (ROPA), providing an up-to-date and accurate view on your companies compliance with standards and regulation.

I look forward to reading Ole’s book to find out more on the power of the meta grid.

In the meantime I was fortunate enough to join Ole and Emma McGrattan on the Enterprise Wide Search podcast to talk about the power of metadata via data contracts. Have a listen and let us know what you think!

Interesting links

Metadata: The New Lifeline for Your GenAI Initiatives by Julia Bardmesser (LinkedIn)

Also on the importance of metadata, this time regarding its importance for GenAI.

5 Best SQL Games to Master Database Skills in 2025

If you’re looking for a way to up your SQL skills these games might be a fun way to do so!

DuckDB is a protocol by Avery Max

Maybe we should be thinking of DuckDB as a protocol, not just a database.

Being punny 😅

Last night’s National Sketching Competition ended in a draw.

Thanks! If you’d like to support my work…

Thanks for reading this weeks newsletter — always appreciated!

If you’d like to support my work consider buying my book, Driving Data Quality with Data Contracts, or if you have it already please leave a review on Amazon.

Enjoy your weekend.

Andrew