Main perf tests #2052

rawdaGastan · 2023-09-11T10:57:26Z

Issues

implement a network test against public nodes #2050

example output

- introduce Task interface - remove testing code - use redis connection from pool - unexpose some methods and fields - update running zos node docs & add example for using perf

- use redis pool correctly - add GetAll method - better naming for tasks with prefix `perf.` - use unix date

rawdaGastan · 2023-09-11T11:43:01Z

waiting for

performance monitor package #2046

- reuse Get code

muhamadazmy · 2023-09-13T11:09:45Z

pkg/graphql/graphql.go

@@ -0,0 +1,68 @@
+// Package graphql for grid graphql support
+package graphql


Did we really needed to implement this from scratch ? can't we use an existing graphql client ?

muhamadazmy · 2023-09-13T11:12:04Z

cmds/modules/noded/main.go

@@ -202,6 +205,41 @@ func action(cli *cli.Context) error {
 		return errors.Wrap(err, "failed to create a new perfMon")
 	}

+	if exists := iperf.Exists(zinit.Default()); exists {


As discussed before running a tests against public nodes happens from ALL nodes (even the ones without public config). The idea is:

Public nodes running iperf running in test mode

All nodes run test against the nodes that has iperf server running

Nodes measure their up/down speed against the random public nodes.

muhamadazmy · 2023-09-13T11:13:31Z

cmds/modules/noded/main.go

+		g, err := graphql.NewGraphQl(env.GraphQL)
+		if err != nil {
+			return errors.Wrap(err, "failed to create a new graphql")
+		}
+
+		freeFarmNodes, err := g.ListPublicNodes(0, 1, true, false)
+		if err != nil {
+			return errors.Wrap(err, "failed to list freefarm nodes from graphql")
+		}
+
+		nodes, err := g.ListPublicNodes(12, 0, true, false)
+		if err != nil {
+			return errors.Wrap(err, "failed to list nodes from graphql")
+		}
+


This is too much to be done by noded. Noded all what it cares about is to register a certain test against the perf framework.

The test ON RUNNING, will pick the nodes to test against. means the nodes can be different each time the test runs

muhamadazmy · 2023-09-13T11:16:29Z

cmds/modules/noded/main.go

+			perfMon.AddTask(&perf.TCPTask{
+				TaskID:    "TCPTask",
+				Schedule:  "* */5 * * * *",
+				ClientIP:  strings.SplitN(node.PublicConfig.Ipv4, "/", 2)[0],
+				Bandwidth: "1M",


While it's correct, but I really don't like this way to construct types. Instead you should create a NewTest function that takes no argument (in this case) because all this stuff should be defined by the test itself, and should be modified by the test plugin (for example iperf test has specific fixed id and schedule)

Also we should have one single test for iperf not 2. It's a single test plugin that will do both tcp and udb (sequentially) and store the result of both tests.

I think a task id can be only iperf. I don't know why it's called a Task here when it's not

The tests also need to run against MANY nodes per test not a single node. I thought this is clear. so a test can take quite some time before it fiinishes (running multiple tests against multiple nodes for tcp and udp

pkg/network/iperf/iperf.go

pkg/perf/monitor.go

pkg/perf/tcp_task.go

muhamadazmy · 2023-09-13T11:33:40Z

pkg/perf/tcp_task.go

+	err = cmd.Run()
+	if err != nil {
+		log.Error().Err(err).Msgf("failed to run iperf tcp task: %s", stderr.String())
+	}
+
+	return out.String(), nil


This is not a good structure. The data from the output need to be parse and put in a structure that describe both upload and download speeds (decent structure) not just raw output from the command.

The structure should also then have information about what node, what Ip the test was run against, did it work or not, what are the speeds, and any other useful information from the test.

pkg/perf/udp_task.go

muhamadazmy · 2023-09-26T08:47:46Z

pkg/perf/iperf_task.go

+
+// NewIperfTest creates a new iperf test
+func NewIperfTest() IperfTest {
+	return IperfTest{taskID: "iperf", schedule: "* */5 * * * *"}


We need to run every 6 hours according to the requirements

pkg/perf/iperf_task.go

pkg/perf/graphql_nodes.go

an * in the second field means "everysecond" so it means to run the test every second of each (6th) our of the day

Omarabdul3ziz and others added 8 commits September 3, 2023 19:26

init performance monitor package

1e99a71

handle new rmb call

7bd548d

resolve comments:

51fc43d

- introduce Task interface - remove testing code - use redis connection from pool - unexpose some methods and fields - update running zos node docs & add example for using perf

resolve comments:

1483702

- use redis pool correctly - add GetAll method - better naming for tasks with prefix `perf.` - use unix date

update go.mod file

91ce438

merge

839c04f

add tcp, udp tests

2cf5bba

get random nodes to test

7ccdd43

rawdaGastan requested review from muhamadazmy, delandtj, MarioBassem, ashraffouda and xmonader as code owners September 11, 2023 10:57

rawdaGastan marked this pull request as draft September 11, 2023 10:57

add freefarm nodes

f5d77ac

rawdaGastan marked this pull request as ready for review September 11, 2023 11:42

Omarabdul3ziz and others added 5 commits September 11, 2023 17:23

- add new handler for getAll method

9369efe

- reuse Get code

Merge branch 'main' into main_perf_package

21b359f

run tasks if it has no results stored, fix getAll method

a79f0c7

merge

52dafa2

log error instead of making noded crash

b2698a1

muhamadazmy changed the base branch from main to main_perf_package September 13, 2023 10:42

muhamadazmy requested changes Sep 13, 2023

View reviewed changes

add iperf and graphql client

ac3df31

Base automatically changed from main_perf_package to main September 26, 2023 08:06

merge

e03b878

muhamadazmy requested changes Sep 26, 2023

View reviewed changes

muhamadazmy reviewed Sep 26, 2023

View reviewed changes

pkg/perf/iperf_task.go Outdated Show resolved Hide resolved

muhamadazmy reviewed Sep 26, 2023

View reviewed changes

pkg/perf/iperf_task.go Outdated Show resolved Hide resolved

muhamadazmy reviewed Sep 26, 2023

View reviewed changes

pkg/perf/iperf_task.go Outdated Show resolved Hide resolved

muhamadazmy reviewed Sep 26, 2023

View reviewed changes

pkg/perf/graphql_nodes.go Outdated Show resolved Hide resolved

rawdaGastan and others added 2 commits September 26, 2023 15:46

fix parse ips and return iperf error itself

b44dcf2

fix cron

ada1c1e

an * in the second field means "everysecond" so it means to run the test every second of each (6th) our of the day

muhamadazmy approved these changes Oct 2, 2023

View reviewed changes

muhamadazmy merged commit 101b2b2 into main Oct 2, 2023
2 checks passed

muhamadazmy deleted the main_perf_tests branch October 2, 2023 07:23

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Main perf tests #2052

Main perf tests #2052

rawdaGastan commented Sep 11, 2023 •

edited

Loading

rawdaGastan commented Sep 11, 2023

muhamadazmy Sep 13, 2023

muhamadazmy Sep 13, 2023

muhamadazmy Sep 13, 2023

muhamadazmy Sep 13, 2023

muhamadazmy Sep 13, 2023

muhamadazmy Sep 13, 2023

muhamadazmy Sep 26, 2023

		@@ -0,0 +1,68 @@
		// Package graphql for grid graphql support
		package graphql

Main perf tests #2052

Main perf tests #2052

Conversation

rawdaGastan commented Sep 11, 2023 • edited Loading

Issues

example output

rawdaGastan commented Sep 11, 2023

muhamadazmy Sep 13, 2023

Choose a reason for hiding this comment

muhamadazmy Sep 13, 2023

Choose a reason for hiding this comment

muhamadazmy Sep 13, 2023

Choose a reason for hiding this comment

muhamadazmy Sep 13, 2023

Choose a reason for hiding this comment

muhamadazmy Sep 13, 2023

Choose a reason for hiding this comment

muhamadazmy Sep 13, 2023

Choose a reason for hiding this comment

muhamadazmy Sep 26, 2023

Choose a reason for hiding this comment

rawdaGastan commented Sep 11, 2023 •

edited

Loading