The case for intentional friction in data platforms

Hello again!

This week I write about how the intentional application of friction in the data platform can guide user behaviour.

There’s also links to a correct-by-design lakehouse, a state of data engineering survey, and anomaly detection at Doordash.

Also a quick reminder, my only in-person workshop this year is happening in June in Belgium. Do join us!

The case for intentional friction in data platforms

I used to constantly talk about how I wanted our data platform to be frictionless.

I wanted it to be so easy to use it would be like an iPhone—smooth, obvious, and keeping out of your way so you can focus on the task you want to do.

But over time I learned that friction isn’t something to avoid at all costs.

In fact, there are times when it’s a necessity.

And when used with intention, friction is a great way to guide user behaviour.

For example, there are many datasets where you can and should ensure gaining access and making use of the data is low friction.

There are also datasets which contain sensitive data, and where the risk of data misuse is high. For these, there should be some friction introduced to make this data harder to access.

That friction signals to the user that there is something different about how they should handle and use this data.

So far, so obvious. But by just doing this, it’s now a binary choice.

But what if through platform features we made it easy to produce anonymised views of data?

These views would have low friction to access and use.

And for many use cases where people say they need access to the full data, they can in fact work with this data. However, they won’t consider this if accessing the full, sensitive data is also low friction, where it’s easy to request access.

If getting access to the full sensitive data has enough friction, then those users will start to consider ways of using that data without having full access.

For example, they will learn they can do local development of pipelines using anonymised data. They don’t need the email addresses, bank details, or other personal data to build and validate this. The anonymised data is fine.

So now we are using friction to guide and change user behaviour, while reducing the risks associated with data misuse.

Interesting links

Building a Correct-by-Design Lakehouse by Weiming Sheng, Jinlang Wang, Manuel Barros, Aldrin Montana, Jacopo Tagliabue, and Luca Bigon

Interesting paper, with a lot of good ideas for data platform design, regardless of whether you want to use their solution or you’re building a lakehouse.

2026 State of Data Engineering Survey by Joe Reis

Some interesting data here. One that caught my eye is that (only?) 10% of respondents have AI embedded in most of their workflows.

Building an anomaly detection platform at DoorDash to catch fraud trends early by Dave Press

Nice, detailed write up on their solution for anomaly detection.

Being punny 😅

I heard that “icy” is the easiest word to spell. Looking at it, I see why.

Thanks! If you’d like to support my work…

Thanks for reading this weeks newsletter — always appreciated!

If you’d like to support my work consider buying my book, Driving Data Quality with Data Contracts, or if you have it already please leave a review on Amazon.

🆕 I’ll be running my in-person workshop, Implementing a Data Mesh with Data Contracts, in June in Belgium. It will likely be only in-person workshop this year. Do join us!

Enjoy your weekend.

Andrew