Running Singer taps and targets with uv

Singer is an open-source framework from Stitch that separates the reading from a source (a tap) and the writing to a data store (a target) by having them communicate with each other using a simple JSON-based protocol.

While a bit dated and without much love from Stitch over the years, I still think it’s a good framework that allows a data engineer to focus on either the reading or the writing of data without worrying about the other side.

However, because taps and targets can be written by different authors, using different dependencies, versions of Python, and so on, running them in a shared environment can be troublesome.

That’s why Singer recommend using a different virtual environment for each.

uv simplifies this, speeds it up, and solves the problem of differing Python versions.

Using uvx you can run any tap or target within its own virtual environment, which uv will create and manage as needed, keeping the dependencies separated.

For example, using the exchange rates tap and the csv target.

uvx tap-exchangeratesapi | uvx target-csv

uv can also use different versions of Python, which is handy as many taps and targets haven’t been updated for a while and may not work on newer versions of Python. You can do that using the --python flag.

uvx --python 3.8 target-csv

All of this is really quick with good use of caching where applicable.

There are a couple of gotchas to watch out for.

One is that the script within the tap may be maned differently than the package, for example with the pipelinewise-target-bigquery. If that’s the case you can use the from flag, as shown below.

uvx --from pipelinewise-target-bigquery target-bigquery

Another is that some taps/targets assume the presence of setup tools. If that’s the case you may get an error like this:

ModuleNotFoundError: No module named 'pkg_resources'

You can solve that by using the --with flag.

uvx --with setuptools target-csv

While Singer is a bit dated and unloved, I still like it as a framework, and now running taps and target is much easier with uv.