iperf3 multiple streams support implementation

[igarny]

Hi Mark,

On this page https://fasterdata.es.net/performance-testing/network-troubleshooting-tools/iperf/multi-stream-iperf3/ ESnet explain about limitations of the iperf3 in running throughput tests with multiple streams.
At the same time that same page explains how this can be achieved. I have not found a ticket about this issue, hence I am putting this here.
Please check the procedure on the page to see about implementation obstacles.
In GEANT, but I believe also WLCG is interested in a iperf3 support for multiple streams.
Adding here @ToniN Antoine as this popped up on the pS Workshop in London

[laeti-tia]

Yes, iperf3 is not multi-threaded (this has been discussed in length at esnet/iperf#289 ) so having multiple sessions (client/server pairs) is the only way to do multiple streams.

And yes, with the 100 Gbps hosts and the embedded devices, there seem to be interest to support multiple streams.

[arlake228]

Just to clarify this issue, what you are asking for is the ability of pScheduler iperf3 tool to have a single run of a task do the following:

1. start multiple iperf3 servers on different ports of the destination side
2. Run multiple clients in parallel to the servers on the source side (possibly using the -T label to help distinguish output as detailed in the doc Ivan shared - really just for parsing convenience so may not be needed)

This is NOT a request to support -P since that is already supported with the parallel option. Just noting this so when I quickly look at this issue in the future I remember what it is actually about.

[laeti-tia]

Yes, correct @arlake228 !

[mfeit-internet2]

If there's a need for better-performing tools, it would be better to develop better-performing tools outside of pScheduler rather than cooking up weird ways to make pScheduler emulate it. The latter has pitfalls.

[igarny]

Hi Mark,

My reading of your response is:
pScheduler manages the measurement tools with a single approach. It will be an overkill to implement different behavior in pScheduler to have this non trivial model of operation for iperf3 measurements with multiple streams.
It should be the tool itself, that should implement this requested model of operation.
Since iperf3 needs to be executed in a specific manner in order to achieve multi-stream type of measurements, then perhaps a "wrapper" solution is a viable option.
The problem with it would be the sustainable control and operation of iperf3.

• events handling from iperf3
• paramaters management of iperf3

If there is ever a solution called iperf3S. Is there a documentation on how to register this tool with pscheduler?

[mfeit-internet2]

If there is ever a solution called iperf3S. Is there a documentation on how to register this tool with pscheduler?

Same as for anything else that runs throughput tests: write a plugin for it. pScheduler doesn't care what goes on under the hood as long as it gets the expected result.

[mfeit-internet2]

I'm iceboxing this; we just added a better-performing throughput tool in the form of Ethr.

Or maybe we can just close it.

[igarny]

This is most likely related to #1157

[mfeit-internet2]

Iperf3 3.16 (#1368) should take care of this.