Direct System Audio API Access

[403access]

💡 Describe the feature

I would like WebAudio to provide a way for web applications to access the underlying system’s native audio APIs—such as CoreAudio (macOS), WASAPI (Windows), and ALSA (Linux)—directly from within the browser. This would allow web applications to bypass the latency and limitations introduced by the browser’s sandboxing mechanisms and leverage the full power of the system’s audio capabilities.

---

🔧 Is there a prototype?

While there is no direct prototype that accesses native APIs due to sandbox restrictions, existing AudioWorkletNode implementations can demonstrate the need for further optimization when working with real-time audio processing. These implementations, even with low-latency settings and WebAssembly integration, show measurable latency (6-10ms or higher) that could be eliminated with direct access to system audio APIs.

---

📋 Describe the feature in more detail

1. 🔌 Native API Hooks: Expose hooks to native audio APIs in the WebAudio API, which could be enabled with user permissions similar to how getUserMedia works for microphone and camera access. This would be particularly useful for professional-grade real-time audio applications, such as live autotune, low-latency voice communication, or music production tools.

2. 🎛️ Selective Native Access: Allow web applications to selectively bypass WebAudio processing for critical audio paths, such as real-time input/output streams. For example, developers could choose whether to process audio in the browser or hand it off directly to the native audio system, depending on the use case.

3. ⏱️ Latency Reduction: The primary goal is to reduce the 6-10ms of browser-added latency. Direct access to system APIs would enable sub-5ms latency, which is crucial for professional real-time audio processing applications (e.g., autotune, live mixing, and real-time voice processing).

4. 🔒 Security Considerations: The system access could be gated behind user permissions, similar to how browsers handle access to hardware like microphones and cameras. Ensuring that this feature does not compromise security while allowing developers greater control over audio processing is critical.

---

This feature would elevate WebAudio's capability for real-time DSP by allowing web apps to match the performance of native audio applications, making the web a viable platform for professional-grade audio tools.

Thanks for your attention and have a great day! 🎉

[403access]

Not sure if understood correctly: Might the following repository related to this feature request?
https://github.com/WebAudio/web-audio-cg

[padenot]

show measurable latency (6-10ms or higher)

our (Firefox) measurements show the opposite, can you share your testing procedure? There is precisely no buffering or anything else going on between the system-level audio callback (regardless of the audio framework/OS) and the AudioWorkletProcessor callback when using Firefox. The only thing that is different from some pro audio software is exactly one process hop for sandboxing reasons, but this causes lower maximum real-time callback load (slightly), not higher latency, under our testing.

One thing that could explain a higher latency than other programs is the fact that the Web Audio API is using a buffer size of 128 frames, and that browers typically open the system-level audio stream at the same size. Improving this (e.g. allowing lower or higher sizes) is tracked is #2450, and the specification text has been written, but it has not been implemented just yet in Web browsers.

[bradisbell]

I think this request is less about the Web Audio API spec, and more about its implementation.

For example, browsers could give us multichannel low latency audio today. Here's one request I submitted for just that, over on the Chromium bug tracker: https://issues.chromium.org/u/2/issues/354625679

[hoch]

There is a prior art based on this concept:
https://github.com/WebAudio/web-audio-cg/blob/master/audio-device-client/explainer.md

However, the Audio WG decided to drop this because the AudioWorklet and new APIs in Web Audio API is capable for serving the needs mentioned above. The only concern that still might be relevant is the round trip latency, but it is an implementation issue, not the spec issue.

For Chromium, the latency issue has been under our radar. You can take a look at https://crbug.com/40125311.