Skip to main content

Source systems must provide applicable data

·1 min

One of our aims with data contracts was to move away from our existing Change Data Capture (CDC) architecture, where the entire database is synced to the data warehouse with exactly the same structure.

This was needed if we were to meet the following goals:

  1. Change management: Software engineers should be able to change their database without impacting downstream consumers
  2. Data applicability: Have the source systems provide data that is immediately applicable

I’ve talked a lot about change management on this newsletter, but the second one is also important.

With data contracts, the data produced by the source system no longer needs to look anything like the source database.

So now the data can be modelled so it:

  1. Meets the requirements of the consumers
  2. Is immediately useful, without joining or other transformation
  3. Represents data using common business language, not service-specific language

That reduces the time to insights, reduces the transformation costs, and make the data useful to a much wider audience, who no longer need to learn about the service just to use the data.

With data contracts, our source systems provide applicable data.


Want great, practical advice on implementing data mesh, data products and data contracts?

In my weekly newsletter I share with you an original post and links to what's new and cool in the world of data mesh, data products, and data contracts.

I also include a little pun, because why not? 😅

Enter your best email here:

    (Don’t worry—I hate spam, too, and I’ll NEVER share your email address with anyone!)

    Andrew Jones
    Author
    Andrew Jones
    I build data platforms that reduce risk and drive revenue.