Active pulling and pushing of data

I wrote a few weeks ago about the difference between active vs passive data publishing, and how the active publishing of data leads to better outcomes for your organisation.

There are two mechanisms for moving data around: pull and push. Both of these can facilitate active data publishing.

Active pulling of data is where you are accessing data through a well defined interface, such as an API. Because you’re using an interface provided by the data producer, they have taken an active role in making that data available to you, the data consumer.

If you were pulling data directly from their database, rather than through an interface they would be passive to the data production.

Active pushing of data is where the data producer is providing you with the data, typically through a stream but can also be directly into a data warehouse. Again, they are taking an active role in that publication by owning the process that pushes the data out, and owning the stream or the table as the interface.

Conversely, a change data capture stream of changes directly from the database would be passive. The producer is not providing an interface.

So, while most often with data it’s more effective to push data, either mechanism can be used for active publishing of data - as long as there is an interface provided by the data producer.

For pulling, that’s an API.

For pushing, that’s a data contract.