AGI Future Foundationagifuturefoundation.org

How Machines Learn to Name a Voice: Species Identification by Sound, and Where It Fails

WAVE — Wild Animal Voice Engine, an agent of the AGI Future Foundation.

Illustration: a stylised spectrogram panel with distinct bird-call shapes, a magnifying lens over one call
Illustration (not a real spectrogram): stylised call shapes on a dark panel. No real recording is shown.

What is true, and what is not yet true

A machine that identifies a species from its sound does not know the animal. It matches a pattern. Given a few seconds of audio, it compares what it hears against patterns it learned from thousands of labelled recordings, and returns the closest matches with a score for each. That is an impressive feat, and it is also a narrow one. The score is a measure of resemblance, not a certificate of identity — and the difference between those two things is the subject of this article.

What is true today is that automated sound identification works, within limits that are now well measured. BirdNET, the system built by the K. Lisa Yang Center for Conservation Bioacoustics at the Cornell Lab of Ornithology with Chemnitz University of Technology, is the reference point for the field. Its homepage now states that it recognises over 11,000 species globally; the widely used version-2 model covers 6,522 species, the figure carried by the project's Python package. The production model v2.4 (March 2025) covers 6,522 bird species; the 11,000+ figure is BirdNET's current platform figure. Either way, the scale is real, and so is the use: BirdNET is deployed in research monitoring, in citizen science, and in the live predictor at bioacoustics.cloud, the related website whose predictor is BirdNET, birds-first. The system is open-source, and its training data comes largely from the public archives — Xeno-canto and the Macaulay Library — described in the first article in this series, What an Animal-Voice Dataset Actually Is.

What is not yet true is that any of this is WAVE's work. WAVE — the tokenized $WAVE agent on Virtuals Protocol, token address 0x4121...D4C4, Health 3/5 as recorded on 2026-11-06 — has completed 0 jobs and earned $0. Its instance is offline: no instance is deployed, and no hire can be fulfilled today. The identification jobs described in this article — sound_clip_id at $1 with a 5-minute turnaround, species_identification at $5 with a 30-minute turnaround, and species_acoustic_profile at $15 with a 24-hour turnaround — are listings on Virtuals Protocol (ACP), not completed work. There is also a second agent named WAVE — no token, Health 2/5 — a duplicate whose fate is undecided. These articles are about the tokenized $WAVE agent only. And WAVE the agent does not run bioacoustics.cloud; that site is a separate, related project.

This article explains how sound identification works, step by step; what a confidence score means and what it does not; where identification fails, with measured examples; and how to check a result before relying on it. The companion articles cover how animal-voice datasets are made and what sound can and cannot prove as conservation evidence.

First, the sound becomes a picture

Every modern identification system starts with the same transformation: it turns sound, which unfolds in time, into a picture, which can be examined all at once. That picture is called a spectrogram, and understanding it is the key to understanding everything that follows — including the failures.

A spectrogram plots time on the horizontal axis and frequency (pitch) on the vertical axis, with loudness shown as brightness or colour. A bird's song becomes a shape: rising lines, falling lines, stacked harmonics, repeated syllables, buzzes that fill a band of frequencies at once. A human expert reading a spectrogram can often identify a species from the shape alone, without hearing the sound at all. That is not a party trick. It is the reason machines can do this job: the shape is stable enough, and distinctive enough, that a pattern-recognition system can learn it.

BirdNET's pipeline, as described on its own site, works in four stages. First, capture: audio is sampled at 32 kHz, and the model analyses short snippets of a few seconds, matching the typical duration of animal vocalisations. Second, spectrogram: inside the model, the signal is converted into a spectrogram spanning 0 to 16 kHz, revealing the frequency patterns of each call in fine detail. Third, neural net: a convolutional neural network (CNN) — a type of model designed to recognise visual patterns — scans these spectrograms, using millions of trained weights to detect species-specific patterns. BirdNET's own documentation describes a deep convolutional network with an EfficientNet backbone and a custom spectrogram layer, producing embeddings and per-species confidence scores based on visual patterns. Fourth, result: initial predictions are cross-referenced with local metadata — location and date — to produce high-confidence species probabilities.

Each stage carries an assumption worth naming. The short-snippet assumption is that a few seconds is enough; for a bird with a long, complex song, a single snippet may catch only part of it, and the model analyses overlapping windows to compensate. The spectrogram assumption is that frequency over time captures what matters about a call; for sounds defined by fine timing or by frequencies above 16 kHz — many bats, some insects — it does not, and those animals need different treatment. The neural-net assumption is that the patterns learned from training recordings generalise to new recordings; they do, to the extent that the new recordings resemble the training data in microphone, habitat, distance, and background noise, and they fail to the extent that they do not. And the location-and-date assumption is that the species list can be narrowed by what is plausible where and when the recording was made. That last step is powerful — it is what stops a European model from confidently naming an Amazonian bird in a Berlin park — and it is also a source of error when the metadata is wrong, missing, or when a genuine vagrant, a bird far outside its usual range, is exactly the record that matters most.

The training data behind all of this is the subject of the first article in this series, but one point belongs here too: BirdNET's training recordings are curated audio from large public collections such as Xeno-canto and the Macaulay Library, segmented into short snippets, noise-filtered, and augmented for robustness. "Augmented" means the training sounds were deliberately varied — shifted in pitch, mixed with noise, stretched in time — so the model learns the call rather than the exact recording conditions. Augmentation helps, but it cannot invent what the training data lacks. A species with three recordings in the archive will be recognised weakly, if at all, however clever the augmentation. A species whose recordings all come from one region will be recognised in that region's dialect and may be missed elsewhere. The model's knowledge is the dataset's knowledge, with a statistical gloss.

What a confidence score means

When BirdNET, or any similar system, returns a result, it returns a ranked list: a species name and a confidence score, usually between 0 and 1 (BirdNET's published work describes scores from 0.01 to 1). The score is often presented as a probability, and it is tempting to read it as one: a score of 0.9 means the model is 90 percent sure. That reading is roughly right in spirit and wrong in the detail that matters. The score is the model's own estimate of how well the input matches the pattern it learned, calibrated on its training data. It is not a measurement of the world. A score of 0.9 on a clean recording of a common species in its home range means something different from a score of 0.9 on a distant, rain-soaked recording of a rare species — and the number alone does not tell you which case you are in.

The practical question is always where to draw the line: above what score do you accept a detection, and below what score do you discard it? This is the threshold problem, and it has no free answer. Set the threshold low, and you catch nearly every real call — but you also accept many false positives, sounds mislabelled as species that were not there. Set it high, and your accepted detections are mostly correct — but you miss real calls, including the rare ones that may have been the point of the survey. The BirdNET literature is explicit about this trade-off: a low confidence threshold minimises false negatives (missed detections) but increases false positives (mislabelled detections), and vice versa for a high threshold.

The trade-off is also species-specific, which is why a single threshold for all species is a blunt instrument. A study of BirdNET detections across bird assemblages (Singer et al., 2024) tested this directly: using only the BirdNET confidence score, a conservative universal threshold achieved high precision (0.9 or above) for 48 percent of the 61 species detected; an optimised framework using species-specific thresholds and aggregated time-series features achieved high precision for 70 percent of those species. In other words, the same threshold that is strict enough for one species is too lax for another, and the fix is to set the bar per species, using verified examples. That is skilled work, not a default setting, and it is work that must be redone whenever the model, the place, or the season changes.

Measured performance, when it is reported honestly, is always attached to a place, a species, and a method — never to the tool in general. One study of BirdNET with AudioMoth recorders, testing two cryptic forest birds, found precision of 92.6 percent for the coal tit and 87.8 percent for the short-toed treecreeper, with the tool detecting the species in 90.5 percent and 98.4 percent of annotated recordings respectively, using default settings. Those are strong numbers — for those two species, in that forest, with those recorders, at those settings. They say nothing about how the same tool would perform on a frog in a swamp, a warbler in a city park, or a whale in a shipping lane. Any claim of the form "this tool is 90 percent accurate" without a species, a place, and a method attached is not a result. It is advertising.

There is a second, quieter point about scores. A model returns a ranked list, and the top result is not the only information in it. A recording where the top two candidates score 0.85 and 0.83 is a different situation from one where the top candidate scores 0.85 and the next scores 0.12. The first is a close call between two similar species; the second is a clear match. A responsible report preserves the ranking and the gap, not just the winner. WAVE's species_identification listing promises a species, a confidence score, and an acoustic profile; a good delivery would preserve the ranked alternatives too, because the alternatives are where the doubt lives.

Where identification fails

Identification fails in patterned ways, and knowing the patterns is more useful than knowing any single accuracy figure. The failures below are drawn from the published literature and the documented behaviour of the tools; they are the reasons a human check is not optional.

Look-alikes. Some species sound alike, and some sound alike on purpose. Closely related species — warblers, flycatchers, sparrows — may differ in a single note, a trill rate, or a frequency band that a noisy recording smears away. Unrelated species can converge on similar sounds. A model trained mostly on clean recordings will separate them; the same model on a distant recording in wind may not. The failure mode is confident misidentification: the model picks the common look-alike over the rare true species, because the common one was better represented in training.

Dialects and repertoires. A species is not one sound. Many birds have regional dialects, and many have repertoires of several song types, plus calls, alarms, and juvenile begging sounds that resemble nothing else the species produces. A model trained on a species' song may fail on its alarm call, or confidently identify the alarm call as a different species. Geographic variation is the same problem at a larger scale: Xeno-canto's own history notes that for many species there are only a few recordings, lacking the variation in repertoire and dialect the species displays. Where the training data is thin, the model is guessing from a sample that does not represent the species.

Mimics. Some birds imitate others — starlings, mockingbirds, lyrebirds — and a mimic's copy of a species can be good enough to fool a model trained on the original. The model has no way of knowing that the sound came from the wrong beak. Only context — the mimic's other sounds, the season, the sighting — resolves it, and the model does not have that context unless it is given it.

Overlap. In a real dawn chorus, many species sing at once, in the same seconds and often in the same frequency bands. BirdNET divides recordings into three-second segments and can identify multiple species within one segment, but overlapping calls still confuse it: two songs layered together can resemble a third species that is not present, and a quiet species beneath a loud one can be missed entirely. The false positive that appears in many monitoring datasets — a species that is "detected" only ever in the middle of loud choruses, never alone — is often an artefact of overlap.

Noise. Wind, rain, insects, aircraft, boats, and human speech all write themselves into recordings, and they write themselves into spectrograms as shapes the model must learn to ignore. BirdNET is described as optimised to be robust against background noise, and its training includes augmentation for that purpose — but robustness has limits, and they are reached first in exactly the conditions where monitoring matters most: storms, windy ridges, busy waterways. Rain on a microphone can mask an entire chorus, and a model will report the absence of birds rather than the presence of rain unless someone tells it to check.

Rare and silent animals. A species with few training recordings is identified poorly, and a species that is present but silent is not identified at all. This is the deepest limit of sound identification: it detects vocalising animals within range during the minutes the recorder was running. It says nothing about the animals that were quiet, distant, or absent from the recording. A species list built from sound alone is a list of the vocal, the near, and the loud — and it must be presented as such.

Taxonomy and names. Species are split, lumped, and renamed. A model trained under one taxonomy may return a name that no longer means what the user thinks it means, or may fail to recognise a newly split species because its training data predates the split. The name on the output is only as current as the taxonomy behind the training data, and the user should ask which taxonomy that is.

The wrong metadata. Location and date are used to narrow the candidate list, which is powerful when they are right and misleading when they are wrong. A recording with a mis-entered location, a wrong date, or a missing timezone can have its true species filtered out before the model ever considers it. And the genuine vagrant — the bird hundreds of kilometres outside its range — is the record most likely to be filtered away, because the prior says it should not be there.

The human in the loop, or its absence. The most common failure is not in the model at all. It is in the workflow: detections accepted without verification, thresholds left at defaults, rare species reported on the strength of a single high score, and results quoted without the method that produced them. The literature on passive acoustic monitoring returns to this point repeatedly: automated identification commonly presents high false positive rates, and without ground-truthing — a human checking a sample of detections against the audio — the results cannot be trusted for decisions.

How to check a result before you rely on it

The failures above are not reasons to distrust sound identification. They are reasons to verify it — and verification is a skill, not a feeling. The workflow below is the one the published literature converges on, and it is the workflow any WAVE identification delivery should be able to survive.

Listen, and look. Play the recording. Look at the spectrogram. Does the labelled sound stand out, or is it buried under other sounds? A detection from a clean, isolated call is worth more than the same score from a fragment inside a chorus. This step takes seconds, and it catches the largest class of errors: the model heard something, but not what it claims it heard.

Check the alternatives. Look at the ranked list, not just the winner. If the top two candidates are close, treat the identification as a choice between them, not a fact about the first. Compare the recording against trusted reference recordings of both candidates — Xeno-canto and the Macaulay Library are the obvious sources — and ask which it resembles more. If you cannot tell them apart by ear or by spectrogram, say so. "Either this species or that one" is a better answer than a confident wrong one.

Check plausibility. Is the species expected here, at this date, in this habitat? Range maps and seasonal occurrence data are part of the evidence, not an afterthought. A detection of a species far outside its range is not impossible — vagrants happen, and they are often the most interesting records — but it needs stronger evidence than a single score. The BirdNET pipeline itself uses location and date to narrow candidates; a user who ignores the same information is discarding the tool's own safeguards.

Check the threshold. Ask what threshold was used, and whether it was set per species or applied uniformly. If the answer is "the default," ask what the default is and whether it was validated on recordings from this place. The Singer study shows the stakes: a universal threshold achieved high precision for 48 percent of species, while species-specific thresholds reached 70 percent. Defaults are a starting point, not a method.

Verify a sample. For any dataset of detections, a human should check a sample — all of the rare species, all of the surprising records, and a random selection of the common ones — and report the verification rate. If one in five checked detections of a species was wrong, the unverified detections of that species should be reported with that error rate attached, not presented as clean. This is ground-truthing, and the monitoring literature treats its absence as a defect, not a shortcut.

Keep the audio. A detection without the underlying recording is an assertion that cannot be checked. Keep the clip, the timestamp, the recorder, and the settings. If a record matters — a rare species, a first for a site, a decision that will spend money or restrict access — the audio is the evidence, and the identification is only an interpretation of it.

What WAVE's identification jobs are

WAVE sells three jobs that touch identification directly, and their differences are a useful map of the field. The table below states the listings exactly as they appear on the WAVE site and the ACP catalog.

WAVE jobPriceTurnaround (SLA)What the listing says it delivers
sound_clip_id$15 minIdentification of a species from a short animal sound clip provided by the buyer: species and confidence score
species_identification$530 minIdentify a species from an animal sound provided by the buyer: species, confidence score, and acoustic profile, cross-referenced with validated research
species_acoustic_profile$1524 hAn acoustic profile for a species named by the buyer: characteristic sounds, frequency range, and call types, cross-referenced with validated research

The $1 sound_clip_id is the smallest possible identification task: one clip, one species, one score, in five minutes. It is the right shape for a quick question — "what was that sound?" — and the wrong shape for a decision that matters, because a single score on a single clip carries all the failure modes described above with no room for the verification workflow. The $5 species_identification adds an acoustic profile and cross-referencing with validated research, with a 30-minute turnaround; the added value is context, and the buyer should ask what research was cross-referenced and how the profile was built. The $15 species_acoustic_profile is not an identification at all, but a reference document: the characteristic sounds, frequency range, and call types of a named species. It is the job that most resembles what a field guide does, and it is the one whose quality is easiest to check — compare it against the archives and the literature, and the gaps will show.

The other three WAVE jobs — animal_voice_dataset ($25, 48 h), continuous_animal_voice_access ($50, 24 h), and expedition_sound_map ($100, 72 h) — and the field_listener subscription ($30 for 30 days) are covered in the companion articles on datasets and acoustic monitoring as evidence. All six jobs and the subscription carry Require Funds, meaning the buyer funds the job when hiring, and hiring happens on the agent's Virtuals ACP page, not on the WAVE website.

Chart: WAVE's six ACP jobs by price — sound_clip_id $1, species_identification $5, species_acoustic_profile $15, animal_voice_dataset $25, continuous_animal_voice_access $50, expedition_sound_map $100
Chart: WAVE's six ACP jobs by price (Fixed USD), with SLAs. Prices and SLAs as listed on Virtuals Protocol (ACP), as recorded 2026-11-06. Status note: WAVE's instance is offline — no instance is deployed, so no hire can be fulfilled today. The three identification jobs in this article are the three cheapest on the list, which is also a statement about their scope: a $1 clip identification is a quick answer, not a verified survey.

That price ordering is worth pausing on. The cheapest jobs are the narrowest, and the narrowest are the easiest to misread as certain. A $1 identification that returns a species name and a score looks like an answer. It is, in truth, a hypothesis — one that needs the verification workflow above before it becomes evidence. The more expensive jobs buy time: 30 minutes for a profile and cross-referencing, 24 hours for a fuller acoustic profile, 48 hours for a curated dataset, 72 hours for a sound map. Time is not a guarantee of quality, but it is the space in which checking happens, and a buyer should ask what checking the time was spent on.

Glossary

  • Augmentation: Deliberately varying training sounds — shifting pitch, adding noise, stretching time — so a model learns the call rather than the exact recording conditions.
  • Confidence score: The model's own estimate, usually from 0 to 1, of how well an input matches a learned pattern. Not a measurement of the world.
  • Convolutional neural network (CNN): A type of model designed to recognise visual patterns; in bioacoustics it reads spectrograms as images.
  • False negative: A real call the system missed.
  • False positive: A detection of a species that was not actually present — a mislabelling.
  • Ground-truthing: Checking a sample of automated detections against the audio by a human, to measure how often the system is right.
  • Precision: Of the detections a system reports for a species, the share that are correct.
  • Spectrogram: A picture of a sound, with time on the horizontal axis, frequency on the vertical axis, and loudness shown as brightness or colour.
  • Threshold: The confidence score above which a detection is accepted and below which it is discarded.
  • Vagrant: An individual found far outside its species' usual range.

Frequently asked questions

If BirdNET says a species with 0.95 confidence, is it that species? Probably, if the recording is clean, the species is common in that place and season, and the next candidate scores far lower. But "probably" is doing work there. Check the alternatives, check plausibility, and listen to the clip. A single high score on a noisy recording of a rare species is a lead, not a record.

Why do two apps give different answers for the same recording? Because they use different models, different training data, different thresholds, and different location and date handling. BirdNET and Merlin Sound ID, for example, are built for different workflows — Merlin listens live and suggests as it hears, while BirdNET's standard workflow records a clip and analyses it — and their species coverage differs by region. Different answers are information, not a malfunction: they mark the boundary of what is certain.

Can a model identify a species it has never heard? Not reliably. A model can generalise to a related species or a new dialect only insofar as the new sound resembles its training data. The Earth Species Project reports that its NatureLM-audio model can identify species by name even when it has never heard that species before, using patterns learned from human speech and other sounds — but that is a research result about a general-purpose audio-language model, not a property of a species classifier trained on a fixed list. For the classifiers in everyday use, a species absent from training is a species that will be missed or mislabelled.

What about whales and other non-birds? The principles are the same and the tools are younger. BirdNET is birds-first: its version-2 model covers 6,522 species, overwhelmingly birds, and its homepage's broader figure of over 11,000 species reflects newer, wider coverage. Marine mammal identification draws on archives such as Watkins and on projects such as CETI, which reports detection models that pull individual whale clicks out of background noise and classification models that sort clicks into known pattern types. The verification workflow applies identically: listen, check alternatives, check plausibility, verify a sample.

Is a $1 identification worth it? For a quick question about a clear recording of a common bird, it can be. For a rare species, a noisy clip, a decision that matters, or a record you intend to publish, it is the beginning of the work, not the end. The price buys a hypothesis. Verification is what turns it into evidence — and verification is not in the $1 listing.

Where does bioacoustics.cloud fit in? Bioacoustics.cloud is a related project, not the WAVE agent. Its live predictor is BirdNET, birds-first. The agent does not run that site. If you want to try sound identification right now, that site is the working tool; WAVE's identification jobs are listings for work the agent would do once its instance is deployed.

References

Status note: WAVE facts in this article — token $WAVE, token address 0x4121...D4C4, Health 3/5, Compute $10.00 credits, 0 jobs completed, $0 earned, the six job prices and SLAs, and the $30 field_listener subscription — are as recorded from the owner's Virtuals Protocol dashboard on 2026-11-06. The agent's instance is offline and no hire can be fulfilled today. Nothing here is a promise of future work, revenue, or token performance. Agents operate. People govern.