Discrepency between realtime/web reporting and the CLI?

[xconverge]

Something weird is going on, where I am able to save a result that makes no sense to me from the web UI, that has a pretty high confidence percentage, and then run it through the birdnet-go cli or BirdNET-Analyzer analyze.py script and get wildly different (much more correct) results than the web UI generated running realtime

If I run https://github.com/kahst/BirdNET-Analyzer/blob/main/analyze.py on these 4 files with the following files, I get the results I expect, 1 valid kestrel and 3 no-matches due to too low of confidence values.

4wavfiles.zip

python3 analyze.py --i ~/Downloads/inputs/ --o ~/Downloads/outputs/ --lat 37.7 --lon -122.4 --week 22 --sf_thresh 0.03 --sensitivity 1 --min_conf 0.7

I get 1 result which is correct, and 3 that are filtered out due to too low of a confidence (which is correct)

Selection	View	Channel	Begin Time (s)	End Time (s)	Low Freq (Hz)	High Freq (Hz)	Common Name	Species Code	Confidence	Begin Path	File Offset (s)
1	Spectrogram 1	1	0	3	0	15000	nocall	nocall	1.0	/Users/xxxxx/Downloads/inputs/actitis_macularius_79p_20240602T154933Z.wav	0

Selection	View	Channel	Begin Time (s)	End Time (s)	Low Freq (Hz)	High Freq (Hz)	Common Name	Species Code	Confidence	Begin Path	File Offset (s)
1	Spectrogram 1	1	0	3	0	15000	nocall	nocall	1.0	/Users/xxxxx/Downloads/inputs/anas_crecca_84p_20240602T152712Z.wav	0

Selection	View	Channel	Begin Time (s)	End Time (s)	Low Freq (Hz)	High Freq (Hz)	Common Name	Species Code	Confidence	Begin Path	File Offset (s)
1	Spectrogram 1	1	0	3.0	0	15000	American Kestrel	amekes	0.9354	/Users/xxxxx/Downloads/inputs/falco_sparverius_89p_20240602T160106Z.wav	0

Selection	View	Channel	Begin Time (s)	End Time (s)	Low Freq (Hz)	High Freq (Hz)	Common Name	Species Code	Confidence	Begin Path	File Offset (s)
1	Spectrogram 1	1	0	3	0	15000	nocall	nocall	1.0	/Users/xxxxx/Downloads/inputs/numenius_phaeopus_81p_20240602T160200Z.wav	0

If I then run birdnet-go CLI on the same files, with what I think are the same parameters in my config, I get the same results

root@88a412241874:/data/inputs# /bin/birdnet-go directory /data/inputs
BirdNET-Go build date: 2024-06-02T17:03:57Z, using config file: /root/.config/birdnet-go/config.yaml
BirdNET-Go build date: 2024-06-02T17:03:57Z, using config file: /root/.config/birdnet-go/config.yaml
Analyzing file: /data/inputs/actitis_macularius_79p_20240602T154933Z.wav
BirdNET GLOBAL 6K V2.4 FP32 model initialized, using 4 threads of available 4 CPUs
Analysis completed in 0 second(s)
Selection	View	Channel	Begin File	Begin Time (s)	End Time (s)	Low Freq (Hz)	High Freq (Hz)	Species Code	Common Name	Confidence
Analyzing file: /data/inputs/anas_crecca_84p_20240602T152712Z.wav
BirdNET GLOBAL 6K V2.4 FP32 model initialized, using 4 threads of available 4 CPUs
Analysis completed in 0 second(s)
Selection	View	Channel	Begin File	Begin Time (s)	End Time (s)	Low Freq (Hz)	High Freq (Hz)	Species Code	Common Name	Confidence
Analyzing file: /data/inputs/falco_sparverius_89p_20240602T160106Z.wav
BirdNET GLOBAL 6K V2.4 FP32 model initialized, using 4 threads of available 4 CPUs
Analysis completed in 0 second(s)
Selection	View	Channel	Begin File	Begin Time (s)	End Time (s)	Low Freq (Hz)	High Freq (Hz)	Species Code	Common Name	Confidence
1	Spectrogram 1	1	/data/inputs/falco_sparverius_89p_20240602T160106Z.wav	00:00:00	00:00:00	0	15000	amekes	American Kestrel	0.9354
Analyzing file: /data/inputs/numenius_phaeopus_81p_20240602T160200Z.wav
BirdNET GLOBAL 6K V2.4 FP32 model initialized, using 4 threads of available 4 CPUs
Analysis completed in 0 second(s)
Selection	View	Channel	Begin File	Begin Time (s)	End Time (s)	Low Freq (Hz)	High Freq (Hz)	Species Code	Common Name	Confidence

so then why did this show up for the actitis_macularius_79p_20240602T154933Z.wav file for example (it is where I copied/downloaded this audio from originally)

The good thing is, the CLI seems to be consistent and match what I saw with the BirdNET-Analyzer repo, but I have no idea why this detection shows up in the web UI and was detected with such a high confidence

Log from realtime birdnet-go at startup:

Starting analyzer in realtime mode. Threshold: 0.75, sensitivity: 1, interval: 15

My only hunch is a potential race condition in the realtime on the PendingDetections where it is getting a confidence from a different detection (of a valid/good bird that exceeds the threshold) and applying it which then allows it through/posts it

If these are both taking in pointers to the queue, I feel like there is a missing lock on the queue between the realtime processor and myaudio.ProcessData() or at the least these should be copies and not pointers to the actual queue that is being written to by myaudio.porcessData()?

func (p *Processor) processDetections(item *queue.Results) {

func (p *Processor) processResults(item *queue.Results) []Detections {

[tphakala]

It is likely due to realtime mode using 1.5 second overlap for analysis, so each 3 second chunk is analysed twice. Add 1.5 overlap for your CLI analyzer commands and see if you get matching results.

[tphakala]

Setting overlap to 1.5 does not produce high confidence detection of sand piper, but setting it to higher values like 2.8 and over push confidence over 75% threshold.

Detection accuracy seem to be very dependent on how bird call is aligned in analysed audio segment.

./birdnet-go file actitis_macularius_79p_20240602T154933Z.wav -t 0.75 -s 1 --overlap 2.99 --locale en
BirdNET-Go build date: 2024-06-02T16:37:51Z, using config file: /etc/birdnet-go/config.yaml
BirdNET-Go build date: 2024-06-02T16:37:51Z, using config file: /etc/birdnet-go/config.yaml
BirdNET GLOBAL 6K V2.4 FP32 model initialized, using 4 threads of available 4 CPUs
Analysis completed in 1 minute(s) 40 second(s)
Selection	View	Channel	Begin File	Begin Time (s)	End Time (s)	Low Freq (Hz)	High Freq (Hz)	Species Code	Common Name	Confidence
31	Spectrogram 1	1	actitis_macularius_79p_20240602T154933Z.wav	00:00:00	00:00:00	0	15000	sposan	Spotted Sandpiper	0.8506
2691	Spectrogram 1	1	actitis_macularius_79p_20240602T154933Z.wav	00:00:00	00:00:00	0	15000	comgre	Common Greenshank	0.8071
3131	Spectrogram 1	1	actitis_macularius_79p_20240602T154933Z.wav	00:00:00	00:00:00	0	15000	comgre	Common Greenshank	0.9202
3281	Spectrogram 1	1	actitis_macularius_79p_20240602T154933Z.wav	00:00:00	00:00:00	0	15000	comgre	Common Greenshank	0.7778
3321	Spectrogram 1	1	actitis_macularius_79p_20240602T154933Z.wav	00:00:00	00:00:00	0	15000	comgre	Common Greenshank	0.8388

[xconverge]

Quoting your message in the other spot to this issue to keep it straight:

"Latest BirdNET range filter model is likely to be defective, there is discussion about it here kahst/BirdNET-Analyzer#296 "

[tphakala]

Range filter model is unrelated to confidence results, these are two separate issues.

[xconverge]

I see, so you are saying just the audio file alone through the model is generating a very high confidence. I assumed since I am seeing some birds that are not possible due to locale, the location and filtering due to location was relevant

[tphakala]

Range filter model is responsible of filtering out species based on location and date (week number), if you are seeing results for species which are not native to your region then problem is in range filtering, could be a threshold issue but also a defect in range filter model, latter is suspected with latest model. I will add option to switch back to previous range filter model.

Description of this issue is about difference in confidence levels during realtime detection and post analysis, that is related to "main" BirdNET AI model.

[xconverge]

I see, the 2nd example in this issue, "anas_crecca" (Definitely not native to my locale in the US) would be a better example for my confusion

I didn't necessarily mean to say or imply "confidence (only)" is different between the CLI and realtime specifically, just that the web UI returns different results than running it from the CLI (with the assumption that how location is used/applied/filtered played a role in this difference as well)

birdnet-go realtime/web gave it an 84% and showed it. Birdnet-go CLI did not return it as a result, I am unsure if this is due to confidence or the range filter.

[phibos]

I have a very similar issue. Real time detection reports something above 80% or even above 90% but with the cli or the BirdNET-Analyzer I'm unable to reproduce the result. I have changed the overlap from 1 to 2.9 in multiple steps but most of the detections I get with the cli are below 20%. When listening to the soundscapes most of them only have white noise.

From 300 perfect detections per day I only have 2 detections with this issue and at the moment I don't know how to help to remove this issue.