Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[feature] Implement Iperf3 check #385 #394

Merged
merged 64 commits into from
Aug 9, 2022
Merged

Conversation

Aryamanz29
Copy link
Member

@Aryamanz29 Aryamanz29 commented Jun 6, 2022

  • Added initial code for Iperf3 check class.
  • Added tests for check class.
  • Added device connection logic.
  • Added charts & metric (TCP & UDP mode).

Steps to test

  1. Make sure your client (openwrt-device) and server both have Iperf3, Openwrt Iperf3 package installed.
  2. Do check the credential section of the device (It must be enabled and working with the right update strategy, i.e., ssh).

173354971-728cb1b4-4bf1-4d89-9c22-ce9b9b1e2bef

  1. In tests/openwisp2/settings.py configure:
OPENWISP_MONITORING_IPERF3_SERVERS = {
    # Running on my local
    # Some Public Iperf3 Servers : https://iperf.fr/iperf-servers.php#public-servers
    # 'be63c4e5-a68a-4650-bfe8-733837edb8be': ['iperf.biznetnetworks.com'],
    'a9734710-db30-46b0-a2fc-01f01046fe4f': ['speedtest.uztelecom.uz'],
    # '<org-pk>': ['<ORG_IPERF3_SERVER>']
}

NOTE : The host can be specified by hostname, IPv4 literal, or IPv6 literal

# for ex
              iperf3 -c iperf.biznetnetworks.com

              iperf3 -c 192.168.5.109

              iperf3 -c 2001:db8::1
  1. Run python3 manage.py run_checks (To run all checks manually).

-------------------------------------------------------------

Demo

iperf_demo.mp4

-------------------------------------------------------------

TCP Charts

Screenshot from 2022-06-24 18-27-43

Screenshot from 2022-06-24 11-51-01

UDP Charts

Screenshot from 2022-06-29 20-31-49

Screenshot from 2022-06-22 13-26-21

Screenshot from 2022-06-22 13-26-25

Closes #385

@coveralls
Copy link

coveralls commented Jun 6, 2022

Coverage Status

Coverage remained the same at 98.709% when pulling 9213aaf on issue-385/iperf-check into e563520 on gsoc22-iperf.

@nemesifier nemesifier changed the base branch from master to gsoc22-iperf June 6, 2022 23:36
Copy link
Member

@nemesifier nemesifier left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I changed the target branch to gsoc22-iperf. Please let's use this from now on.

CC: @pandafy @okraits.

@Aryamanz29 Aryamanz29 self-assigned this Jun 10, 2022
@Aryamanz29 Aryamanz29 added the enhancement New feature or request label Jun 10, 2022
@Aryamanz29 Aryamanz29 force-pushed the issue-385/iperf-check branch 4 times, most recently from 335cf4f to bad881d Compare June 14, 2022 07:46
@Aryamanz29 Aryamanz29 force-pushed the issue-385/iperf-check branch from f968f79 to 2b474ae Compare June 20, 2022 13:22
Copy link
Member

@pandafy pandafy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work @Aryamanz29! 👏🏼 👏🏼

@nemesisdesign @okraits things that I have tested manually:

  • The iperf check is created automatically when a new device registers
  • The iperf check is not created if OPENWISP_MONITORING_AUTO_IPERF is set to False.
  • The iperf check execute for a device having an active SSH connection.
  • The iperf check does not execute for a device whose SSH connection is in not_working state
  • The charts are shown for the gathered data.

Edge cases to handle

  • The database reports that DeviceConnection is working, but the management tunnel goes down just before iperf check was executed. The code should be able to gracefully handle this and also notify users that SSH connection is not working. We already have a mechanism for sending notifications. @Aryamanz29 please check other checks for reference and ask us your doubts.

Things to discuss in next weekly meeting

  • What parameter should trigger alert for iperf check?
  • Which data should be shown to users with charts?

openwisp_monitoring/check/settings.py Outdated Show resolved Hide resolved
openwisp_monitoring/check/classes/iperf.py Outdated Show resolved Hide resolved
openwisp_monitoring/check/classes/iperf.py Outdated Show resolved Hide resolved
openwisp_monitoring/check/classes/iperf.py Outdated Show resolved Hide resolved
openwisp_monitoring/check/classes/iperf.py Outdated Show resolved Hide resolved
openwisp_monitoring/check/classes/iperf.py Outdated Show resolved Hide resolved
openwisp_monitoring/check/classes/iperf.py Outdated Show resolved Hide resolved
openwisp_monitoring/check/classes/iperf.py Outdated Show resolved Hide resolved
openwisp_monitoring/tests/test_selenium.py Outdated Show resolved Hide resolved
tests/openwisp2/settings.py Show resolved Hide resolved
@Aryamanz29
Copy link
Member Author

Aryamanz29 commented Jun 23, 2022

@pandafy In 8823581 I have fixed #399, Now you can see below that error is longer produced because earlier we're executing command without considering the device_connection working status just right after connect() method.

One more thing, do we really need to notify users for this event? I mean when I'm testing it locally, the "SSH not working" alert already popped out & it already fulfilled our purpose, what do you think? Although, I have added logger.warn() which is specific for the check.

Screenshot from 2022-06-23 15-21-42

Screenshot from 2022-06-23 15-21-46

@Aryamanz29 Aryamanz29 force-pushed the issue-385/iperf-check branch from 016ea9f to 8823581 Compare June 23, 2022 10:23
@pandafy
Copy link
Member

pandafy commented Jun 23, 2022

One more thing, do we really need to notify users for this event? I mean when I'm testing it locally, the "SSH not working" alert already popped out & it already fulfilled our purpose, what do you think? Although, I have added logger.warn() which is specific for the check.

As the user is already getting one notification for SSH not working, we don't need to create another notification. Logging a warning is sufficient.

I will do manual testing later today.

Copy link
Member

@pandafy pandafy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Aryamanz29 I did a manual test of the changes and I don't see an error for #399. I will do an in-depth code review tomorrow.

Good work @Aryamanz29! 👏🏼

Copy link
Member

@pandafy pandafy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nemesisdesign @okraits I need some clarity on how we are planning to store the data in the InfluxDB. The current implementation ignores the bitrate from the UDP test, which seems wrong to me.

Thinking out loud
A good argument could be "because UDP bitrate would be similar to TCP bitrate so we can ignore that". If so, then why bother doing a UDP test? Is it only for getting jitter and packet loss?

openwisp_monitoring/check/classes/iperf.py Outdated Show resolved Hide resolved
openwisp_monitoring/check/classes/iperf.py Outdated Show resolved Hide resolved
openwisp_monitoring/check/classes/iperf.py Outdated Show resolved Hide resolved
openwisp_monitoring/check/classes/iperf.py Outdated Show resolved Hide resolved
openwisp_monitoring/check/classes/iperf.py Outdated Show resolved Hide resolved
openwisp_monitoring/db/backends/influxdb/queries.py Outdated Show resolved Hide resolved
tests/openwisp2/settings.py Show resolved Hide resolved
openwisp_monitoring/check/classes/iperf.py Outdated Show resolved Hide resolved
@Aryamanz29 Aryamanz29 force-pushed the issue-385/iperf-check branch 3 times, most recently from 4bf1fd9 to 1d94482 Compare June 24, 2022 12:59
Aryamanz29 and others added 23 commits August 9, 2022 17:11
- Added ping config checks in celery beat configuration.
- If args is None executes all checks.
- Added username, password, rsa-public-key-path parameter
to DEFAULT_IPERF_CHECK_CONFIG.

Closes #414
- Changed iperf auth logic (RSA key in params instead of path).
- Added IPERF_CHECK_RSA_KEY_DELETE (If true RSA will be delete from the device).
- Added IPERF_CHECK_RSA_KEY_PATH (To set default iperf RSA public key).
- Impoved existing tests as per review suggestion.
- Connected data points with new bar+lines chart type.
- Added fill(0) to the chart query to fill blank data points(ie.N/A).

Closes #417
- Combined all iperf settings into one setting.
- Removed default from OPENWISP_MONITORING_IPERF_CHECK_CONFIG
- The order of getting params for iperf check is:
check_params(through admin panel)-> OPENWISP_MONITORING_IPERF_CHECK_CONFIG-> DEFAULT_IPERF_CHECK_CONFIG
- Updated iperf tests.

Closes #418
- Added bitrate param to tcp and udp mode.
- Improved iperf tests.
- Improved alert_on_related_field tests.
- Found that options.mode = 'lines+markers' for the
stackedbar+lines charts which streches charts x-axis
to some backward time we can use options.mode = 'lines'
for iperf charts to avoid this.

Fixes #422
- Combined transfer and bandwidth charts.
- improved docs.
- Removed alert_on_related feature from this PR.
@Aryamanz29 Aryamanz29 force-pushed the issue-385/iperf-check branch from 5f4c51f to 9213aaf Compare August 9, 2022 11:42
@nemesifier nemesifier merged commit dc0316a into gsoc22-iperf Aug 9, 2022
@nemesifier nemesifier deleted the issue-385/iperf-check branch August 9, 2022 12:39
@Aryamanz29 Aryamanz29 changed the title [feature] Implement Iperf check #385 [feature] Implement Iperf3 check #385 Oct 7, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[feature] Create an Iperf3 check class
4 participants