Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

samples: Add USB Audio to broadcast audio source #64601

Merged

Conversation

larsgk
Copy link
Contributor

@larsgk larsgk commented Oct 31, 2023

Adds support for using a connected host to
broadcast audio via USB Audio.

Offload LC3 encoding to separate thread.

@larsgk larsgk force-pushed the add_usb_broadcast_audio_source branch from abdb9b2 to b91d76d Compare October 31, 2023 07:31
@tmon-nordic tmon-nordic self-requested a review October 31, 2023 07:43
@larsgk larsgk marked this pull request as ready for review October 31, 2023 10:35
@larsgk
Copy link
Contributor Author

larsgk commented Oct 31, 2023

Moved the PR out of draft to get some initial reviews. There are some glitches on the USB Audio side (periodic approx 10 secs of clean flow and audio, then 2-3 secs of missing data coming from USB) which may require updates to how the USB data flow is handled.

samples/bluetooth/broadcast_audio_source/src/main.c Outdated Show resolved Hide resolved
samples/bluetooth/broadcast_audio_source/src/main.c Outdated Show resolved Hide resolved
samples/bluetooth/broadcast_audio_source/src/main.c Outdated Show resolved Hide resolved
samples/bluetooth/broadcast_audio_source/src/main.c Outdated Show resolved Hide resolved
samples/bluetooth/broadcast_audio_source/src/main.c Outdated Show resolved Hide resolved
samples/bluetooth/broadcast_audio_source/src/main.c Outdated Show resolved Hide resolved
samples/bluetooth/broadcast_audio_source/src/main.c Outdated Show resolved Hide resolved
@larsgk
Copy link
Contributor Author

larsgk commented Oct 31, 2023

Moving the PR to draft until I have the encoder in a separate thread and addressed the issues mentioned in comments above

@larsgk larsgk marked this pull request as draft October 31, 2023 12:47
Copy link
Contributor

@tmon-nordic tmon-nordic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The USB issues are related to following remark from nRF5340 Product Specification:
"If the EasyDMA transfer on the isochronous endpoint is not completed before the next SOF event, the result of the transfer is undefined."

lc3_encode_delaying_usbd_workqueue

Because lc3_encode() can clearly execute for longer than 1 ms, it is completely unacceptable to have it executing in cooperative thread context.

samples/bluetooth/broadcast_audio_source/src/main.c Outdated Show resolved Hide resolved
@larsgk
Copy link
Contributor Author

larsgk commented Oct 31, 2023

ok - offloaded to a separate thread and everything flows smoothly now (cleaning up the PR). There are a very few occasional glitches, probably because we don't have sample rate conversion to compensate for clock drifting/mismatch - but rare enough for the sample IMO. Working on reintroducing multi stream (stereo) and the rest of the change requests.

@larsgk larsgk force-pushed the add_usb_broadcast_audio_source branch 3 times, most recently from 99a7c2f to c9039ea Compare November 6, 2023 08:32
@larsgk larsgk marked this pull request as ready for review November 6, 2023 08:32
samples/bluetooth/broadcast_audio_source/src/main.c Outdated Show resolved Hide resolved
samples/bluetooth/broadcast_audio_source/src/main.c Outdated Show resolved Hide resolved
samples/bluetooth/broadcast_audio_source/src/main.c Outdated Show resolved Hide resolved
samples/bluetooth/broadcast_audio_source/src/main.c Outdated Show resolved Hide resolved
samples/bluetooth/broadcast_audio_source/src/main.c Outdated Show resolved Hide resolved
samples/bluetooth/broadcast_audio_source/src/main.c Outdated Show resolved Hide resolved
samples/bluetooth/broadcast_audio_source/src/main.c Outdated Show resolved Hide resolved
@larsgk larsgk force-pushed the add_usb_broadcast_audio_source branch 4 times, most recently from c6cf525 to 492fe1c Compare November 6, 2023 13:04
@larsgk larsgk force-pushed the add_usb_broadcast_audio_source branch 2 times, most recently from 212b3e5 to f8a7708 Compare November 6, 2023 15:11
@larsgk
Copy link
Contributor Author

larsgk commented Nov 7, 2023

@Thalley and @tmon-nordic - it seems to be in a stable state now with the tests I made. Would you be able to re-review and possibly accept further improvements (e.g. to USB corner case handling) to come as a follow-up?

samples/bluetooth/broadcast_audio_source/src/main.c Outdated Show resolved Hide resolved
samples/bluetooth/broadcast_audio_source/src/main.c Outdated Show resolved Hide resolved
samples/bluetooth/broadcast_audio_source/src/main.c Outdated Show resolved Hide resolved
samples/bluetooth/broadcast_audio_source/src/main.c Outdated Show resolved Hide resolved
printk("Initialized ring buf %zu: capacity: %u\n", i,
ring_buf_capacity_get(&(streams[i].audio_ring_buf)));
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
printk("Initialized ring buf %zu: capacity: %u\n", i,
ring_buf_capacity_get(&(streams[i].audio_ring_buf)));
printk("Initialized ring buf %zu: capacity: %u\n", i,
ring_buf_capacity_get(&(streams[i].audio_ring_buf)));

@larsgk larsgk force-pushed the add_usb_broadcast_audio_source branch from f8a7708 to 62150b2 Compare November 8, 2023 06:15
@larsgk larsgk requested a review from Thalley November 8, 2023 06:17
@larsgk larsgk requested a review from tmon-nordic November 8, 2023 07:04
@larsgk larsgk force-pushed the add_usb_broadcast_audio_source branch from 62150b2 to 2f6d2e4 Compare November 8, 2023 07:09
@larsgk larsgk force-pushed the add_usb_broadcast_audio_source branch from 2f6d2e4 to 3d341a2 Compare November 8, 2023 07:34
@larsgk
Copy link
Contributor Author

larsgk commented Nov 8, 2023

(last push was just commit message fix)

@larsgk larsgk force-pushed the add_usb_broadcast_audio_source branch from 3d341a2 to 281f9fb Compare November 8, 2023 15:00
Adds support for using a connected host to
broadcast audio via USB Audio.

Offload LC3 encoding to separate thread.

Signed-off-by: Lars Knudsen <[email protected]>
@larsgk larsgk force-pushed the add_usb_broadcast_audio_source branch from 281f9fb to 175eff1 Compare November 8, 2023 15:28
Copy link
Collaborator

@Thalley Thalley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good enough as is, with some improvements that can be applied

Comment on lines +34 to +35
default n
depends on ENABLE_LC3
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rather than doing

CONFIG_USE_USB_AUDIO_INPUT=y

in samples/bluetooth/broadcast_audio_source/boards/nrf5340_audio_dk_nrf5340_cpuapp.conf, you could follow the approach above and do

default y if SOC_NRF5340_CPUAPP

So that is is applied for both the regular and audio kit

Comment on lines +3 to +5
# Use USB Audio as input
CONFIG_USE_USB_AUDIO_INPUT=y
CONFIG_USB_DEVICE_PRODUCT="Zephyr Broadcast Source"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be interesting to find a way to set CONFIG_USB_DEVICE_PRODUCT="Zephyr Broadcast Source" for any device that enables USB, and not just the nRF5340 ADK

Comment on lines +265 to +267
if (streams[i].lc3_encoder == NULL) {
printk("ERROR: Failed to setup LC3 encoder - wrong parameters?\n");
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we return here, or allow only setting up a subset of the encoders?

@larsgk
Copy link
Contributor Author

larsgk commented Nov 13, 2023

@tmon-nordic could you check if you could approve? GH still tells me you have a change requested. If anything, could it be a followup PR?

@tmon-nordic tmon-nordic dismissed their stale review November 13, 2023 10:12

LC3 processing was moved to preemptible thread, resolving USB issues

@tmon-nordic
Copy link
Contributor

@tmon-nordic could you check if you could approve? GH still tells me you have a change requested. If anything, could it be a followup PR?

I just dismissed the review because the USB issue is fixed. The problem was simply preventing USBD workqueue from processing events within the deadline (SOF = 1 ms). The PR looks fine to me, but I am not really the correct person to approve it.

@carlescufi carlescufi merged commit 78714b1 into zephyrproject-rtos:main Nov 14, 2023
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants