[SKIP SOF-TEST] Simple dependency-free ALSA test rig for PCM capture analysis.

[andyross]

Just drop this script on a test device to run it. No tools to build, no dependencies to install. Confirmed to run on Python 3.8+ with nothing more than the core libraries and a working libasound.so.2 visible to the runtime linker.

When run without arguments, the tool will record from the capture device for the specified duration, then emit the resulting samples back out the playback device without processing (except potentially to convert the sample format from s32_le to s16_le if needed, and to discard any channels beyond those supported by the playback device).

Passing --chirp-test enables a playback-to-capture latency detector: the tool will emit a short ~6 kHz wave packet via ALSA's mmap interface (which allows measuring and correcting for the buffer latency from the userspace process) and simultaneously loop on short reads from the capture device looking for the moment it arrives.

Passing --echo-test enables a capture-while-playback test. The script will play a specified .wav file ("noise.wav" by default) for the specified duration, while simultaneously capturing, and report the "power" (in essentially arbitrary units, but it's linear with actual signal energy assuming the sample space is itself linear) of the captured data to stdout at the end of the test.

[andyross]

From thesofproject/sof#8681 , see there for previous discussion

[marc-hb]

A good few, minor codestyle issues left otherwise LGTM but we need a reviewer who knows ALSA well.

Unlike other repos, we run pylint in this one and most of these few warnings seem valid and easy to fix:
https://github.com/thesofproject/sof-test/actions/runs/7393258262/job/20112906366?pr=1144

Please use pylint: disable=W1234 for the ones you can't or don't want to fix.

[kv2019i]

Thanks @andyross ! While there is obvious overlap with alsabat and rest of alsa-utils, the "drop-in" capability will for sure have value in many cases.

[plbossart]

is there any reason to restrict the playback usage to S16_LE?

[andyross]

It's all any of the devices I'm using have the capability to emit, basically. Agreed that a more general tool would be nice, but there's cost to code size too.

[plbossart]

it's fine to keep S16_LE, it's just not clear if this was intentional due to lack of time or by design due to XYZ restrictions.

[plbossart]

doesn't work for 44.1kHz

[andyross]

Not understanding, rate/1000 gives you the number of samples per millisecond. It's true that 44100 doesn't divide evenly, but the code doesn't care. This is just a conveniently small buffer size to do the analysis with later.

[plbossart]

all I am saying is that there should be a clear point on when the code no longer works.
We've have countless issues with such rounding that work fine in configuration A and break in configuration B.

[andyross]

You'd be OK with an assert that the number of frames is >zero? I could add a "chunk_size" tunable I guess too, I just don't see much value there.

Again, the specific size of these buffers isn't important. All the analysis happens on lists of them and doesn't care about how many samples each buffer contains. They're kept small to (1) provide reasonably fine-grained timestamp behavior, since we can only timestamp between API calls, and (2) avoid doing a big allocation and buffer copy synchronously with the capture, since this is python and I don't want to deal with performance headaches.

[plbossart]

not following which latency you are measuring, does this not add playback+capture continuous latencies together?

[andyross]

Actually what I'm really measuring is a whitebox test inside AEC, which is doing essentially the same thing this python code is doing to watch the chirp go by. This is here as a general smoke test from userspace. But to be specific: it's measuring the time between the arrival of the chirp signal at the head of the userspace mmap buffer to the return from snd_pcm_readi() that contains it.

[plbossart]

Not following your explanation, sorry.

[andyross]

1. Fill the mmap playback buffer[1] with silence
2. Start playing
3. Spin until there is just enough space in the buffer to accept the chirp, write it
4. Check buffer stats and take timestamp.

Now we know exactly how many samples are "ahead" of the chirp but still in host memory, so we can subtract them from the measured time it takes the chirp to appear on the capture stream. You can't do this with snd_pcm_writei() as the buffering is internal to the userspace API.

[1] Which is obviously the actual DMA buffer, though the API doesn't promise this.

[plbossart]

not all DMAs provide a precise position, it's a recurring problem with audio - and it's only made worse with the bursty nature of the fimware-controlled DMAs.

You probably have at least one ms of error each way for the Intel HDaudio hw. no idea for others.

[andyross]

No doubt. But we do what we can. The point was just to not have an unmeasurable 0-85ms of latency in the userspace process, or to have to guess at "how long after the first writei() call does the DMA stream start running", etc... These numbers aren't the whole story, but they're at least reliable.

FWIW: I'm getting a pretty consistent 30-31ms round trip on MTL with AEC enabled, which seems not totally unreasonable (10ms buffer for AEC alone, plus 0-1ms getting into and out of the DP thread due to scheduling cadence mismatch) but also that it probably has room for improvement. mt8195 is a disaster in comparison: 100ms on the device I'm using, which AFAICS isn't in the firmware at all but in some hardware component (you can also hear it squash short chirp bursts, which is why the default length is so large).

[kv2019i]

I think this is ok with suitable disclaimers. E.g. the 30-31ms number sounds in the right ballpark and matches data got (without AEC) on with ALSA latency tester https://www.alsa-project.org/wiki/Test_latency.c and jack-delay https://wiki.archlinux.org/title/Professional_audio#Latency_verification . Both tools have more dependencies (so don't fit the usage this tool aims to cover), but can be used on any Linux setup to verify the numbers (for a given platform).

pylint count low enough

[marc-hb]

Found a few more nits + easy fixes for 2 pylint warnings but Python codestyle looks great.

Need other people (@plbossart , @aiChaoSONG , @singalsu , @ranj063, others?) to review the audio logic and ALSA API usage.

[marc-hb]

Then what happens?

[andyross]

You have to write ctypes code directly, e.g. the one struct definition in the class here, or the fact that the return value of functions ending with "name" is forced to be a c_char_p (the default is a return value of "int", which breaks because it chops off the top bits of the pointer). Ctypes isn't super clean as a FFI, but it's not so bad. But 90% of ALSA code in this script looks just like ALSA code in C, with the advantage that the errno returns become exceptions.

[marc-hb]

Suggested change

	occasional constructed integer or byref() pointer.
	occasional constructed integer or byref() pointer, e.g. pcm_channel_area_t below.

I read too fast and didn't realize this script has already sample solutions for this.

[marc-hb]

Kill four birds with one stone:

• get rid of the pylint warning
• make echo_test() more generic
• make opts less global
• do not add one level of indentation to the entire echo_test() code

Suggested change

	echo_test()
	with open(opts.noise, "rb").read()[HEADER_LEN:] as buf:
	echo_test(buf)

[andyross]

That looks clever, but interestingly this doesn't work. It turns out that the "with" syntax isn't integrated with the internal reference counter at all[1], it's done at the class level by forcing the value assigned to implement __enter__/__exit__ methods. A file object has that, but a bytes object doesn't (because it's a heap block handled by the interpreter refcount!). So the result is "AttributeError: __enter__"

Really that "with" warning is a false positive. What it's detecting is the call to open(). But the object gets used and discarded in just the one line of code ("with" requires enter/exit, but the actual file descriptor gets closed out of the refcounter), so there's nothing to clean up. To get this right, pylint would need to do lifetime analysis of the result of open() and not just detect that it's called, and it's not that smart.

[1] Which, sigh, is very much on brand for python. All about top-level API definitions and never about clean integration with a core interpreter engine.

[marc-hb]

I guess the following could have worked (the previous suggestion was hasty copy/paste) but I understand it is a false positive in the first place.

with open(opts.noise, "rb") as reader:
    echo_test(reader)

[marc-hb]

Same as before: code style looks good, someone else must approve the audio and ALSA API parts

[marc-hb]

I guess the following could have worked (the previous suggestion was hasty copy/paste) but I understand it is a false positive in the first place.

with open(opts.noise, "rb") as reader:
    echo_test(reader)

[kv2019i]

I'm good with this. In majority of cases, we'll be using aplay/arecord et al, so I don't think this has a be a perfect copy. But I do think this can be a very valueble tool on constrained environments and I can see value in having this available in sof-test.

[marc-hb]

@plbossart any objection?