--- title: "Data contracts and the meta grid" date: 2025-06-20 description: "This weeks post is all about metadata, and how data contracts feeds into the meta grid.\n\nThere's also links to the importance of metadata to GenAI, SQL games, and thinking of DuckDB as a protocol." categories: ["Data Contracts","Metadata","Meta Grid"] source: https://andrew-jones.com/newsletter/2025-06-20-data-contracts-and-the-meta-grid/ --- # Data contracts and the meta grid Hello from a [scorching England](https://www.bbc.co.uk/news/articles/cm2z4rmyl0yo)! This weeks post is all about metadata, and how data contracts feeds into the meta grid. There's also links to the importance of metadata to GenAI, SQL games, and thinking of DuckDB as a protocol. --- ## Data contracts and the meta grid Over the last few years people have slowly realised the importance and the power of metadata, i.e. the data that describes the data. Someone who has been particularly vocal on this trend is [Ole Olesen-Bagneux](https://www.linkedin.com/in/ole-olesen-bagneux/), author of the upcoming book [Fundamentals of Metadata Management](https://www.oreilly.com/library/view/fundamentals-of-metadata/9781098162818/), who [defines metadata as follows](https://substack.com/inbox/post/151452674): > Metadata is a description that is both attached with what is described and placed somewhere else, to make what is described discoverable and manageable. I like this definition, and it aligns nicely with how I think of data contracts. A data contract is simply a collection of metadata that describes the data. A simple idea, but because of the power of metadata, it itself becomes powerful. The data contract is *attached* to the data, in that it is defined alongside the code that generates the data and therefore will stay updated and in sync with the data as the data itself evolves. But the metadata in the data contract is also *placed somewhere else* to drive something else. ![Diagram showing the data contract being defined with the data-generating code/system and then used to drive the datastore, data tooling, and a data catalog.](https://andrew-jones.com/images/newsletter/2025-06-20/Data-contracts-driving-tools-and-catalogs.png) For example we might expose this metadata in a data catalog so users can easily find and access data across the organisation. And we might also place this metadata in tooling to implement data platform capabilities such as data retention and access management. And so on. We can place it wherever we need to to deliver some capability or other value to the producers and/or the consumers of the data. Importantly, the *source of truth* for this metadata remains the data contract, but it's value is realised by having that metadata made available elsewhere. Ole has gone further and coined the term [meta grid](https://searchingfordata.com/) as he considers how the power of other metadata, often imperfect, incomplete, and unstandardised across an organisation can be unlocked. Here, the data contract becomes one source of metadata that can be combined with others to drive processes or other value. One example I can think of would be to combine the metadata from a data contract with metadata on your data governance policies to create a [Record of Processing Activities (ROPA)](https://ico.org.uk/for-organisations/uk-gdpr-guidance-and-resources/accountability-and-governance/accountability-framework/records-of-processing-and-lawful-basis/), providing an up-to-date and accurate view on your companies compliance with standards and regulation. ![Diagram showing the data contract together with other metadata powering a record of processing activities. Also other metadata being combined with the data contract and/or further metadata to drive other, unnamed processes.](https://andrew-jones.com/images/newsletter/2025-06-20/Data-contracts-and-the-meta-grid.png) I look forward to reading Ole's book to find out more on the power of the meta grid. In the meantime I was fortunate enough to join Ole and [Emma McGrattan](https://www.linkedin.com/in/emma-mcgrattan/) on the [Enterprise Wide Search podcast](https://podcast.actian.com/enterprise-wide-search-09-andrew-jones-the-power-of-data-contracts) to talk about the power of metadata via data contracts. Have a listen and let us know what you think! --- ## Interesting links **[Metadata: The New Lifeline for Your GenAI Initiatives](https://www.linkedin.com/pulse/metadata-new-lifeline-your-genai-initiatives-julia-bardmesser-gj5mf/) by Julia Bardmesser (LinkedIn)** Also on the importance of metadata, this time regarding its importance for GenAI. **[5 Best SQL Games to Master Database Skills in 2025](https://www.sqlnoir.com/blog/games-to-learn-sql)** If you're looking for a way to up your SQL skills these games might be a fun way to do so! **[DuckDB is a protocol](https://www.summer.io/blog/duckdb-is-a-protocol) by Avery Max** Maybe we should be thinking of DuckDB as a protocol, not just a database. --- ## Being punny 😅 Last night's National Sketching Competition ended in a draw. --- ## Thanks! If you'd like to support my work... Thanks for reading this weeks newsletter — always appreciated! If you'd like to support my work consider buying my book, [_Driving Data Quality with Data Contracts_](https://data-contracts.com), or if you have it already please leave a review [on Amazon](https://amzn.to/3Nm4lN9). Enjoy your weekend. Andrew