Make framework configurable to run hierarchical suite of workloads

[ganesank-git]

The user should be able to provide a folder, subfolder or a test name to run.

Example:
./run.sh -t dataplane
./run.sh -t dataplane/ai/classification/resnet50
./run.sh -t dataplane/ai

[ndn97]

added -c option which takes folder/sub-folder that contains test names,

implementing folders in testnames (-t) option will not have clear identifcation of path/testnames, this change doesnt breaks old way of running.. some tests are given as follows

##to run list of tests in custom path
ndn@issubuntu01:~/k-bench$ ./run.sh -c ./dataplane -t "custom_ai ai"
Running test custom_ai found in ./dataplane and results redirected to "./results_run_11-Apr-2021-09-54-57-pm/custom_ai"
Starting benchmark, writing logs to results_run_11-Apr-2021-09-54-57-pm/custom_ai/kbench.log...

##if custom path is not provided , default custom path ./config dir is taken (original behavior)
ndn@issubuntu01:~/k-bench$ ./run.sh
Usage: ./run.sh -r [-t -c -o ]
Example: ./run.sh -r "kbench-run-on-XYZ-cluster" -t "cp_heavy16,dp_netperf_internode,dp_fio" -o "./"

Valid test names:

all || all_control_plane || all_data_plane ||
Since no tests specified, I am running the default workload: config/default
Running test default found in ./config/ and results redirected to "./results_run_11-Apr-2021-09-55-55-pm/default"
Starting benchmark, writing logs to results_run_11-Apr-2021-09-55-55-pm/default/kbench.log...
Running workload, please check kbench log for details...

##original behavior of the tests,works
ndn@issubuntu01:~/k-bench$ ./run.sh -t dp_redis
Running test dp_redis found in ./config/ and results redirected to "./results_run_11-Apr-2021-09-57-09-pm/dp_redis"
Starting benchmark, writing logs to results_run_11-Apr-2021-09-57-09-pm/dp_redis/kbench.log...

original behavior, run all control plane tests

ndn@issubuntu01:~/k-bench$ ./run.sh -t all_control_plane
Running test default found in ./config/ and results redirected to "./results_run_11-Apr-2021-09-58-16-pm/default"
Starting benchmark, writing logs to results_run_11-Apr-2021-09-58-16-pm/default/kbench.log...

original behavior, run all control plane tests

ndn@issubuntu01:~/k-bench$ ./run.sh -t all_data_plane
Running test dp_fio found in ./config/ and results redirected to "./results_run_11-Apr-2021-09-59-50-pm/dp_fio"
persistentvolumeclaim/fio-block-pvc created

[ganesank-git]

Hi Nagesh, thanks for the change. I understand why you think we should have two separate options for test name and config directory. But, in the case of an ask where the user has multiple tests he wants to run in a particular directory, the user still have to use the top level directory as the test name. My suggestion would be to remove test names altogether and just use one directory name. If the directory has a config.json, it will be run. If not, we recursively look at the subdirectories for config.json files and execute them one by one.