The iPhone model for integrated platforms

Hey friends 👋

This week I write about how we can use the iPhone model to create integrated developer, data, and other platform features, and the change in internal developer platforms that need to make this happen.

Also links to articles on data as code, data platforms for data scientists, and stream and batch analytics with Iceberg.

The iPhone model for integrated platforms

One of the problems with data platforms is how separate they are to the internal developer platforms being created for software engineers.

This separation means data platform teams have to spend time rebuilding a lot of the primitives they need, for example CD/CD pipelines, IaC pipelines, and so on.

Worse, this separation leads to a poor user experience for users of the platform - particularly those who are using some capabilities from the internal developer platform and the data platform.

In my talk at the London Platform User Group this week I emphasised how important it was to meet users where they are, using the same tooling and patterns they are already used to, to reduce the friction and the barrier to adoption of data platform tooling - in this case, of using data contracts.

We were able to do that because we could build our tools into the internal developer platform that was provided by our platform engineering team.

We also benefitted from the primitives they had built, including the tooling around Jsonnet†, cloud resource configuration with Config Connector, easy deployment of our “sidecar” services to Kubernetes for data management, and more, so we could focus just on the capabilities we were implementing.

I didn’t appreciate how lucky we were to have that until I watched the other talk at this meetup by Abby Bangser, who spoke about how platforms need to be explicitly designed to support this, so teams like a data platform team can provide these specialist capabilities for their users.

The analogy Abby made was the iPhone.

The iPhone has a few built-in apps that everyone needs - phone, messages, camera, etc.

But it’s real power is the app store, which has specialist apps filling any niche.

Just look at the finalists for the 2025 App Store Awards announced this week and you’ll see an app for helping musicians record and mix tracks, a fantastically detailed app for keeping track of everything in your home, an app that empowers user who are blind or have low vision, and so on.

Apple supports developers building these and all the other apps by providing developer environments (Xcode), frameworks (UIKit, AVFoundation, Core Data, etc), and community events (WWDC and many smaller events).

Developer platforms need to be built in the same way, allowing specialist capabilities to be provided by data teams, security teams, finance teams (e.g. for FinOps), etc, and supporting them as first-class users/developers of the platform.

Now, our internal developer platform was not explicitly designed in this way.

However, our platform team were supportive of us building within their platform. And, in another piece of luck, our data platform team had some great software engineers who didn’t mind getting their hands dirty in someone else’s code and working out how to add the capabilities we needed to add.

In the future we shouldn’t need these lucky circumstances to come together.

Platforms will be built explicitly to enable this.

The meetup wasn’t recorded, but you can watch Abby’s 15m keynote from KubeCon + CloudNativeCon to learn more.

† Jsonnet was our configuration language of choice, which we also adopted for our data contracts. Again, meeting them where they are.

Interesting links

Data as Code: From ETL to Executable Truth by Jim Grafton

Interesting thoughts on where data transformations could go if we codify it.

Beyond the Hype: How Data Platforms Address Real Blockers for Data Science by Anna Bergevin

Data platforms sit as the bridge between producers and consumers. AI/ML use cases represent some of the highest value opportunities in many enterprises - data platforms can help bridge the gaps between producers and consumers, increase stability, provide context, and generally facilitate improved collaboration and utilization of our company’s data.

This is why we do what we do :)

How Would You Like Your Iceberg Sir? Stream or Batch Ordered? by Jack Vanlightly

Interesting comparison of different approaches to steam and batch analytics with Apache Iceberg, and the trade-offs of each approach.

Being punny 😅

We’ve had a data breach incident, but I couldn’t get my hands on our security guy. Maybe he ransomware?

Thanks! If you’d like to support my work…

Thanks for reading this weeks newsletter — always appreciated!

If you’d like to support my work consider buying my book, Driving Data Quality with Data Contracts, or if you have it already please leave a review on Amazon.

🆕 Also check out my self-paced course on Implementing Data Contracts.

Enjoy your weekend.

Andrew