Skip to content

Commit

Permalink
Tools: testbench: Update README.md text after IPC4 switch
Browse files Browse the repository at this point in the history 
The document is updated after changing build default to IPC4
and to show usage of added helper scripts.

Signed-off-by: Seppo Ingalsuo <[email protected]>
  • Loading branch information
@singalsu
singalsu committed Nov 1, 2024
1 parent 6739d5e commit 258f823
Showing 1 changed file with 155 additions and 59 deletions.
214 changes: 155 additions & 59 deletions tools/testbench/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -19,76 +19,143 @@
The simplest way to build and execute testbench is with supplied
scripts. It executes a simple chirp waveform test for a number of
processing components. Corrupted audio or failure in execution
results to fail of test.

The commands "build-tools.sh -t" and "host-testbench.sh" are currently
valid only for default IPC3 build of testbench.
or a memory violation results to fail of test.

```
cd $SOF_WORKSPACE/sof
scripts/build-tools.sh -t
scripts/build-tools.sh
scripts/rebuild-testbench.sh
scripts/host-testbench.sh
```

### Manual run of IPC3 testbench
### Manual run of IPC4 testbench

As an example, process a wav file and listen it. The example
processing component is DC block. The wav file is first converted with
sox to raw 32 bit 48 kHz stereo format, then processed by testbench,
converted back to wav.
processing component is IIR equalizer. The wav file is first converted
with sox to raw 32 bit 48 kHz stereo format, then processed by
testbench, converted back to wav.

The example processes the playback direction of topology
sof-hda-benchmark-eqiir32.tplg with IIR component. The playback
direction in the component test topologies consists of host pipeline 1
and dai pipeline 2. Therefore the command line has option -p 1,2.

```
cd $SOF_WORKSPACE/sof
sox --encoding signed-integer /usr/share/sounds/alsa/Front_Left.wav -L -r 48000 -c 2 -b 32 in.raw
tools/testbench/build_testbench/install/bin/testbench -r 48000 -R 48000 -c 2 -n 2 -b S32_LE \
-t tools/build_tools/test/topology/test-playback-ssp5-mclk-0-I2S-dcblock-s32le-s32le-48k-24576k-codec.tplg \
-i in.raw -o out.raw
tools/testbench/build_testbench/install/bin/sof-testbench4 -r 48000 -c 2 -b S32_LE \
-p 1,2 -t tools/build_tools/topology/topology2/development/sof-hda-benchmark-eqiir32.tplg \
-i in.raw -o out.raw
sox --encoding signed-integer -L -r 48000 -c 2 -b 32 out.raw out.wav
aplay out.wav
```

The testbench binary can be debugged and profiled with native
executable tools like gdb, ddd (GUI for gdb), gprof, and valgrind.
To test capture use -p 3,4 with same topologies. To run both
directions, use -p 1,2,3,4 and provide multiple input and output
files separated with comma. Use e.g. -i i1.raw,i2.raw
-o o1.raw,o2.raw.

### Profiling of testbench run on xtensa
### Run testbench with helper script

Fist, the testbench is build in this example for Intel MTL
platform. This needs access to Cadence Xplorer toolchain with the core
confiration for the platform.
The scripts/sof-testbench-helper.sh simplifies the task. See the help
text shown with option -h.

```
Usage: scripts/sof-testbench-helper.sh <options>
Options
-b <bits>, default 32
-c <channels>, default 2
-h shows this text
-i <input wav>, default /usr/share/sounds/alsa/Front_Center.wav
-k keep temporary files in /tmp
-m <module>, default gain
-n <pipelines>, default 1,2
-o <output wav>, default none
-p <profiling result text>, use with -x, default none
-r <rate>, default 48000
-t <force topology>, default none, e.g. production/sof-hda-generic.tplg
-v runs with valgrind, not available with -x
-x runs testbench with xt-run simulator
Example: run DRC with xt-run with profiling (slow)
scripts/sof-testbench-helper.sh -x -m drc -p profile-drc32.txt
Example: process with native build DRC file Front_Center.wav (fast)
scripts/sof-testbench-helper.sh -m drc -i /usr/share/sounds/alsa/Front_Center.wav \
-o ~/tmp/Front_Center_with_DRC.wav
Example: check component eqiir with valgrind
scripts/sof-testbench-helper.sh -v -m eqiir
```

The above manual run can be executed with single command. The -i could
be omitted as well since the ALSA clip is the default input wav.


```
scripts/sof-testbench-helper.sh -m eqiir -i /usr/share/sounds/alsa/Front_Center.wav -o out.wav
```

To run the same example with Xtensa simulator (MTL platform), set up
the build environment, build, and execute with additional -x
option. An MCPS estimate for pipeline execution is printed in the end
of run than can be useful when checking impact of code
optimizations. Note that the cycles of testbench file component that
replaces host-copier and dai-copier are excluded.

```
cd $SOF_WORKSPACE/sof
export XTENSA_TOOLS_ROOT=~/xtensa/XtDevTools
export ZEPHYR_TOOLCHAIN_VARIANT=xt-clang
scripts/rebuild-testbench.sh -p mtl
source tools/testbench/build_xt_testbench/xtrun_env.sh
scripts/sof-testbench-helper.sh -x -m eqiir -i /usr/share/sounds/alsa/Front_Center.wav -o out.wav
```

Next the testbench is run with xt-run simulator. The example component
is dynamic range processing (DRC). For proling data generate use
option --profile. Note that this is a lot slower than native run. An
1s extract of a pink noise file is used as example.
### Run Xtensa profiler with helper script

When profiling add to above run script option -p, e.g. (can omit output wav conversion).


```
sox --encoding signed-integer /usr/share/sounds/alsa/Noise.wav -L -r 48000 -c 2 -b 32 in.raw trim 0.0 1.0
$XTENSA_PATH/xt-run --profile=profile.out tools/testbench/build_xt_testbench/testbench \
-q -r 48000 -R 48000 -c 2 -n 2 -b S32_LE \
-t tools/build_tools/test/topology/test-playback-ssp5-mclk-0-I2S-drc-s32le-s32le-48k-24576k-codec.tplg \
-i in.raw -o out.raw 2> trace.txt
scripts/sof-testbench-helper.sh -x -m eqiir -p profile-eqiir.txt
less profile-eqiir.txt
```

Then convert the profiler data to readable format and check it.
The output of xt-gprof looks like

Note: Current version of testbench does not have functional quiet mode
"-q" switch to suppress trace. Due all debug traces print
out. Majority of performance is spent in printing.
```
Flat profile:
self total
cumulative self cycles cycles
% cycles cycles calls /call /call name
(K) (K) (K) (K)
56.73 10144.51 10144.51 137088 0.07 0.07 iir_df1
12.91 12453.23 2308.72 1428 1.62 8.72 eq_iir_s32_default
3.23 13031.59 578.36 4290 0.13 3.61 module_adapter_copy
2.22 13429.03 397.44 2860 0.14 0.67 file_process
2.14 13811.37 382.34 3178 0.12 0.13 memmove
```
$XTENSA_PATH/xt-gprof tools/testbench/build_xt_testbench/testbench profile.out > example_profile.txt
less example_profile.txt

To profile a set of processing components in SOF there is example
script sof-testbench-build-profile.sh. It profiles also two
hda-generic topologies with multiple SOF components in it. The help
for the script is shown with option -h:

```
Usage: scripts/sof-testbench-build-profile.sh <options>
-d <directory> directory to place code profiling reports
-h shows this text
-p <platform> sets platform for scripts/rebuild-testbench.sh
```

It can be run for platform mtl with command
```
scripts/sof-testbench-build-profile.sh -p mtl
```

The profiling reports and component run logs (with MCPS) can be found
from default directory tools/testbench/profile.

### Perform audio processing quality checks

This step needs Matlab or Octave tool with signal processing
Expand All @@ -105,50 +172,79 @@ octave -q --eval "pkg load signal io; [n_fail]=process_test('eq-iir', 32, 32, 48
See from interactive Octave shell command "help process_test" the
explanation for the test run parameters.

### Manual run of IPC4 testbench
### Manual run of IPC3 testbench

Apply this patch to SOF and rebuild it.
The testbench build is by default IPC4. The IPC version can be changed by applying next
patch:

```
diff --git a/src/arch/host/configs/library_defconfig b/src/arch/host/configs/library_defconfig
index d150690aa..6d167dfba 100644
index 1b9483ffe..5c944d292 100644
--- a/src/arch/host/configs/library_defconfig
+++ b/src/arch/host/configs/library_defconfig
@@ -21,8 +21,8 @@ CONFIG_COMP_VOLUME=y
@@ -22,8 +22,8 @@ CONFIG_COMP_VOLUME=y
CONFIG_COMP_VOLUME_LINEAR_RAMP=y
CONFIG_COMP_VOLUME_WINDOWS_FADE=y
CONFIG_DEBUG_MEMORY_USAGE_SCAN=n
-CONFIG_IPC_MAJOR_3=y
-CONFIG_IPC_MAJOR_4=n
+CONFIG_IPC_MAJOR_3=n
+CONFIG_IPC_MAJOR_4=y
-CONFIG_IPC_MAJOR_3=n
-CONFIG_IPC_MAJOR_4=y
+CONFIG_IPC_MAJOR_3=y
+CONFIG_IPC_MAJOR_4=n
CONFIG_LIBRARY=y
CONFIG_LIBRARY_STATIC=y
CONFIG_MATH_IIR_DF2T=y
```

Then process a wav file and listen. The example processing component
is DC blocker. The wav file is first converted with sox to raw 32 bit
48 kHz stereo format, then processed by testbench, converted back to
wav.
As an example, process a wav file and listen it. The example
processing component is DC block. The wav file is first converted with
sox to raw 32 bit 48 kHz stereo format, then processed by testbench,
converted back to wav.

```
cd $SOF_WORKSPACE/sof
sox --encoding signed-integer /usr/share/sounds/alsa/Front_Center.wav -L -r 48000 -c 2 -b 32 in.raw
tools/testbench/build_testbench/install/bin/testbench -r 48000 -R 48000 -c 2 -n 2 -b S32_LE -p 1,2 \
-t tools/build_tools/topology/topology2/development/sof-hda-benchmark-dcblock32.tplg \
scripts/rebuild-testbench.sh
scripts/build-tools.sh -t
sox --encoding signed-integer /usr/share/sounds/alsa/Front_Left.wav -L -r 48000 -c 2 -b 32 in.raw
tools/testbench/build_testbench/install/bin/sof-testbench3 -r 48000 -R 48000 -c 2 -n 2 -b S32_LE \
-t tools/build_tools/test/topology/test-playback-ssp5-mclk-0-I2S-dcblock-s32le-s32le-48k-24576k-codec.tplg \
-i in.raw -o out.raw
sox --encoding signed-integer -L -r 48000 -c 2 -b 32 out.raw out.wav
aplay out.wav
```

The difference in command line is the use of other topology for IPC4
and use of -p command line option to select pipelines 1 and 2 from
this topology for scheduling and connecting to input and output files.
In this topology the host playback pipeline is 1 and the dai playback
side pipeline is 2. Capture side pipelines would be 3 and 4. And
running both playback and capture would need more input and output
files to be added to -i and -o as comma separated. The more advanced
use cases simulations with IPC4 are still under work.
The testbench binary can be debugged and profiled with native
executable tools like gdb, ddd (GUI for gdb), gprof, and valgrind.

### Manual profiling of testbench run on xtensa

The example is with IPC3 build for Intel MTL platform. The similar commands would work with IPC4
build also, with changed topology for IPC4. This needs access to Cadence Xplorer toolchain with
the core confiration for the platform.

The debugging and profiling of IPC4 testench is similar as with IPC3.
```
cd $SOF_WORKSPACE/sof
export XTENSA_TOOLS_ROOT=~/xtensa/XtDevTools
export ZEPHYR_TOOLCHAIN_VARIANT=xt-clang
scripts/rebuild-testbench.sh -p mtl
source tools/testbench/build_xt_testbench/xtrun_env.sh
```

Next the testbench is run with xt-run simulator. The example component
is dynamic range processing (DRC). For proling data generate use
option --profile. Note that this is a lot slower than native run. An
1s extract of a pink noise file is used as example.

```
sox --encoding signed-integer /usr/share/sounds/alsa/Noise.wav -L -r 48000 -c 2 -b 32 in.raw trim 0.0 1.0
$XTENSA_PATH/xt-run --profile=profile.out tools/testbench/build_xt_testbench/testbench \
-r 48000 -R 48000 -c 2 -n 2 -b S32_LE \
-t tools/build_tools/test/topology/test-playback-ssp5-mclk-0-I2S-drc-s32le-s32le-48k-24576k-codec.tplg \
-i in.raw -o out.raw 2> trace.txt
```

Then convert the profiler data to readable format and check it.

```
$XTENSA_PATH/xt-gprof tools/testbench/build_xt_testbench/testbench profile.out > example_profile.txt
less example_profile.txt
```

0 comments on commit 258f823

Please sign in to comment.