We favour moving quickly over reliability
·1 min
Data engineers typically favour moving quickly over reliability.
Data Engineering
2024
Deliberately model your data
If you’re creating data, you’re modelling data.
Computation is the most expensive part of your data stack
·1 min
Computation is the most expensive part of your data stack.
Even if you had perfect data, you should still be building defensive data pipelines
·1 min
I talk a lot about data quality and how it could be improved, because I strongly believe that with a little bit more discipline we can do a lot better.
Being boring
An article that has stayed with me since it was published back in 2015 is Dan McKinley’s Choose Boring Technology.
Self serve doesn't stop at providing data
·1 min
You can’t do self-serve just by providing datasets that you understand, but your users don’t.
What do you want from your data?
·1 min
What do you want from your data?
Count the joins
·1 min
I enjoyed this post from Nicole Radziwill, PhD on LinkedIn:
Data engineering provides the foundations for FinOps
·1 min
Following on from yesterdays post about measuring for reporting and measuring for action, one data-driven initiative I’m working on is our FinOps program.
Measure for reporting, then measure for action
Recently I’m finding it useful to split some of the data-driven initiatives I’m working on into two parts: measure for reporting, and measure for action.
Want great, practical advice on implementing data mesh, data products and data contracts?
In my weekly newsletter I share with you an original post and links to what's new and cool in the world of data mesh, data products, and data contracts.
I also include a little pun, because why not? 😅
Enter your best email here:
(Don’t worry—I hate spam, too, and I’ll NEVER share your email address with anyone!)