Daily data contracts tips

Since Hadoop came along in 2006 and significantly reduced the cost of storing “big data” we’ve often been focused on how much data we can bring in centrally, with the assumption that we’ll use it to create value later.

I mentioned yesterday that one way to reduce the amount of data transformations (and the costs of them) is to challenge the assumption we need to bring all data centrally before it can be useful.

I enjoyed this post by Paul McMahon on how all code is technical debt. Paul argues that the more code an application has, the slower the development, due to the assumptions that exist in that code and the features they support. As such, he views all code as technical debt.

As I wrote yesterday, many data professionals don’t trust the data they are building on. And many users of data and data applications don’t trust the data they’re being provided.

At most of my recent talks I’ve asked the audience - who are made up of data professionals - a simple question: Do you trust your data?

Most of the problems we’re solving in our organisations are not unique to us:

• We need a way to store data
• We need a way to discover data
• We need a way to transform data
• We need a way to build and present dashboards
• We need a way to train and deploy machine learning models

And so on.

No matter where you are in your organisation, you have the ability to change it.

Often things are as they are because they always have been.

So challenge the assumption that they have to be that way.

One way to stand out from your colleagues and advance your career is to find or create some at-work extracurricular opportunities.

Written communication is such an important skill for your career.

Most of us do most of our professional communication on something like Slack or Teams, particularly as we spend more time working remotely. The quality of that communication affects how well you’ll work with others and the relationships you build with them.