Bacpipe: a tool to compare bioacoustic machine learning methods

Bacpipe, short for bioacoustic collection pipeline, is a software tool that I have been working on since the beginning of my PhD. The idea was to create an easy-to-use tool that allows us to compare different bioacoustic machine learning models. (To avoid confusion: while referring to the simplistic machine learning models, I am speaking of deep learning neural networks).

Much like the instrument bagpipe, one thing gets put in and a slightly overwhelming — but, let’s be honest, hauntingly beautiful — amount of stuff gets put out. In the case of bacpipe, we input some audio data and get those slightly overwhelming but hauntingly beautiful embeddings. But what actually are embeddings?

To explain, I will first start by motivating why embeddings are hauntingly beautiful. Machine learning models get trained in different ways. The most common variant of training in machine learning is called supervised learning. It’s the one most of us are familiar with – on a conceptual level. We pair data and labels and train the algorithm to correctly classify the data according to the labels. The problem with that is: once you’ve decided on your labels and trained the algorithm, it’s incapable of recognizing any other labels. In bioacoustics that means you’re never able to use BirdNET because it was trained on bird sounds but you have a dataset of frog sounds.

But is there a way to compare if BirdNET or other models are capable of differentiating between your frog sounds? Yes! And embeddings are the key to that question.

All bioacoustic machine learning models generate embeddings. Embeddings are just long lists of numbers – also known as vectors. Embeddings are the models’ way of encoding information. That long list of numbers then gets mapped onto the labels (or in technical terms classes) that the model was trained on. But if you just ignore that last step, and instead work with the embeddings, you get one vector for every 3 seconds of audio in your input data (BirdNET works with 3-second windows, other models use different lengths).

So if we now generate embeddings for BirdNET as well as for Perch (Google’s machine learning model trained on a lot! of bird sounds) we have two sets of vectors/embeddings, that we can compare.

Each of the colors corresponds to a species of frog recorded by Juan Sebastian Cañas (also a doctoral candidate in this network) and his co-authors in the tropical rain forests of South America. And while this colorful set of points allows us to now compare any publicly available bioacoustic model and process things like clustering and classification analysis to see which model is better or worse for differentiating between specific sounds.

And the great thing is that bacpipe can do that for you. All you need to do is specify where your sounds are located, change the configurations and run the pipeline. Depending on the models you selected to compare, bacpipe will generate embeddings for each one of them. And on top of that you get a dashboard which visualizes everything.

If you have annotations for your dataset, computing classification and clustering is already supported by default. But even if you just have a big messy dataset with no annotations, you can still use bacpipe and use it to see diurnal, seasonal or other patterns. Bacpipe does this by automatically extracting the date and time metadata from the filenames. In the figure below you can see the BirdNET embeddings color-coded by time-of-day on the left and by their deployment location on the right. The points are exactly the same but by color-coding them in this way, we can see that BirdNET seems to organize the data by deployment location. While this does not replace a thorough analysis of the data, it might highlight some patterns, that could otherwise be overlooked.

Bacpipe is intended as a collaborative project and I am more than happy for others to contribute to this software tool to have it grow into something useful for as many of us as possible. How to download, configure and use bacpipe is explained on the github repository. Give it a try, you can find it here.