Preloading sounds have become noticeable in Chrome

[NeilFraser]

We need to preload and cache all the sounds, otherwise when we play a sound the first instance will play a second late. The Audio API has a 'preload' attribute that's designed to provide exactly this. However, when tested back in 2014, this did not work across all browsers. So we chose to play all sounds at minimum volume upon page load. At a volume of just 1%, this didn't used to be audible.

However, this preloading is now audible. Crank your volume up, and load any of the demo pages in Chrome (at least on a Mac). On page load, one can clearly hear the click and delete sounds. Despite the code, there's no way that this is 1% volume.

I think it's time that Blockly take another look at sounds and determine if there are better ways of preloading sounds in the majority of browsers. We will probably need to keep the hacky low-volume preload technique for IE, but at a minimum I'd expect that Chrome and FF should have proper preloading abilities. Also be aware that iOS can apparently only play one sound. This is a swamp of browser-specific gotchas.

Surfaced due to google/blockly-games#171

[samelhusseini]

Let's look at using .preload = "auto". That leaves it up to the browser to decide whether or not to preload the audio file. It seemed to work quite well in the browsers I tested (Chrome, Firefox, Safari), but we should look at how some older browsers (that we still support) deal with that.

https://www.w3schools.com/tags/att_audio_preload.asp

[moniika]

A related issue. The current approach for preloading sounds seems to not work in firefox.
The preloading is not audible, warning is seen: "Autoplay is only allowed when approved by the user, the site is activated by the user, or media is muted", and a delay is heard for sounds.

[samelhusseini]

I spent some time looking into this, and it seems adding the preload:auto attribute to the HTMLAudioElement is preferred way to notify the browser that we'd like an audio file preloaded, without actually enforcing it. If the browser has other reasons to ignore our "request" for preloading it, then so that seems fair to me.

Changing to this would require cross browser physical testing. I tried this on IOS, and it appears we're not handling click events quick enough thus not allowing us to play the audio (as now browsers require you to have an associated click even to play audio).

[tuzz]

We're also seeing that audio events are sometimes not being played in our Blockly workspace on iOS devices. It does seem to be the case that they're not being handled quickly enough and that is then triggering the browser's restriction that audio must be played in response to a user action.

In addition to preloading the audio, it might also help to wrap this call to mySound.play() in the same defensive code that is used above - although ideally, the audio should always work.

[jeroenvekemans]

We see the same behavior testing https://blockly.games/maze on iOS devices:

1. at startup, the preload of one audio is noticable at normal volume
2. the connect sound when you attach one block to another is not played at all on these devices

Are there any related issues/updates on this?

[cpcallen]

I can verify that on iOS (specifically, Chrome on iOS 14 on iPhone SE 2016) the only sound that plays is the preloading click sound; no other sounds play at all AFAICT (so for example Pond is entirely silent while running).

Earlier @NeilFraser wrote:

Despite the code, there's no way that this is 1% volume.

Possibly true, but be aware that if 1% volume means 1% of amplitude that that is only -20dB; given that human hearing has a dynamic range of ~120dB that' not really particularly quiet. (Or to put it another way: a decrease in amplitude to 1% only quiets 7 bits of the typical 16 bits per sample.)

[Entkenntnis]

This is happening in Firefox 112 for me. I can hear the preloading click - it's noticably quiter (possibly 1%), but still audible.

Can you set the volume to an even lower value?

blockly/core/workspace_audio.ts

Line 92 in 817ffab

sound.volume = 0.01;