Blogs

As someone who works in an enablement team, I’m trying to instil a best practice of answering with docs. This has a number of benefits to both us and our users:

Over on the GoCardless Tech Blog I’ve written a post about how we’re improving data quality with Data Contracts. Check it out and let me know what you think!

Data is at the heart of every meaningful service, and it’s the effective use of that data which builds a product, and a business.

Almost all data platforms start with a change data capture (CDC) service to extract data from an organisations transactional databases - the source of truth for their most valuable data. That data is then transformed, joined, and aggregated to drive analysis, modelling, and other downstream services.

We recently had an incident with our data pipeline, resulting in data being lost on route to our data platform. Of course, you never want an incident, but failures are a fact of life. What’s important is how you prepare for them and respond to them, and in that sense this was a great incident.

Inspired by Richard Seroter, I want to publicly share some of the things I’d like to learn this year.

There’s a trend in the industry to make data science and machine learning more accessible, allowing engineers to build and deploy standard models without needing to have a strong data science background. Examples include BigQueryML (standard models in SQL), Ubers Ludwig and h2o.ai.

I’ve been a Tech Lead for a few years now, though I’d say I’ve only been a good Tech Lead for about a year. So what, exactly, does a good Tech Lead do?

As I start to think about some of the upcoming projects we’ll be working on over the next year and how we might go about building them, I wanted to consider where lambda architecture fits in our toolbox for building data services.

Postmortems are a well established process followed in the aftermath of an incident. Often the most visible output from the process is a structured document, with some or all of the following components: