From 9aeec3df08e63b9565586df44b3989602c5af042 Mon Sep 17 00:00:00 2001 From: Andy Lake Date: Wed, 3 Apr 2024 18:46:27 -0400 Subject: [PATCH] 5.1.0b1 release notes draft 1 --- _releasenotes/2024-04-07-5.1.0b1.md | 166 ++++++++++++++++++++++++++++ 1 file changed, 166 insertions(+) create mode 100644 _releasenotes/2024-04-07-5.1.0b1.md diff --git a/_releasenotes/2024-04-07-5.1.0b1.md b/_releasenotes/2024-04-07-5.1.0b1.md new file mode 100644 index 0000000..2ae938a --- /dev/null +++ b/_releasenotes/2024-04-07-5.1.0b1.md @@ -0,0 +1,166 @@ +--- +version: 5.1.0b1 +title: "perfSONAR 5.1.0 Beta 1 Release Notes" +supported: true +--- + +Released {{ page.date | date: "%B %-d, %Y"}} + +Features and Highlights +------------------------ + +### New Grafana Interface + +The perfSONAR Toolkit and MaDDash interfaces have been replaced with [Grafana](https://www.grafana.com)! Grafana is a popular visualization platform which is familiar to many of our users. It has a range of plugins that make it easier to display perfSONAR data in useful ways, allows for greater customization and opens the door for integration with other data sources. + +When you install or upgrade a perfSONAR Toolkit, the default web interface for displaying results will automatically use Grafana. The installation process will automatically install and configure Grafana for you. It will also provide you with a default set of dashboards that display your results and give details about your perfSONAR host, so no requirement to build dashboards on your own. + +We also now offer a way to create MaDDash-like displays in Grafana using a new [pSConfig Grafana Agent](https://docs.perfsonar.net/release_candidates/5.1.0/psconfig_grafana_agent.html). If you have an existing MaDDash installation then it will reamin untouched but we are encouraging everyone to move to the new system. A "cookbook" for setting-up a central OpenSearch Archive with Grafana can be found [here](https://docs.perfsonar.net/release_candidates/5.1.0/cookbook_central_archive.html). + +### Threaded iperf3 Support + +iperf3 3.16 with multi-threading is now fully supported by pScheduler. In general, this allows measurement to run at higher throughput than previously possible with single-threaded iperf3. You can leverage this feature with the `--parallel` option of `pscheduler task throughput`. + +### Python pSConfig + +pSConfig had been completely re-implemented in Python (formerly in Perl). This will make support easier going forward and is part of a larger effort over the next few releases to transition away from the last remnants of Perl in perfSONAR. There are no changes to the pSConfig template format, this is purely a source code change and should be mostly transparent. + +### Enhanced perfSONAR Instrumentation + +perfSONAR installations now export a greater number of stats about the host system and perfSONAR applications such as pScheduler and pSConfig. This is done using Prometheus node_exporter and some custom perfSONAR endpoints. Information from these endpoints is collected periodically, stored in Opensearch and displayed in the default Grafana dashboards. The new pSConfig also has a host metrics agent than can centrally collect this information from multiple hosts and display them centrally (details in the [Central Grafana Cookbook](https://docs.perfsonar.net/release_candidates/5.1.0/cookbook_central_archive.html)) + +### Better Troubleshooting of Archiving Issues + +perfSONAR now includes a `psarchive troubleshoot` command to help pinpoint issues when there are problems storing results. It will check connectivity to components like OpenSearch and Logstash, try to fetch data and verify credentials. Along with enhancements to `pscheduler troubleshoot` and the previously mentioned instrumentation, this a part of an ongoing effort to make it easier to pinpoint problems with your perfSONAR installation. + +### Updated OS Support +The offically supported operating systems for 5.1.0 are: + +- Alma and Rocky Linux 9 (other EL variants may work, but at your own risk) +- Debian 11 +- Debian 12 +- Ubuntu 20 +- Ubuntu 22 + +5.1.0 will NOT be released for CentOS 7/EL7, Debian 10 or Ubuntu 18 given all are at or close to their end-of-life at time of release. + +*Note: Support for EL8 is being explored and may be offered by final release on a limited basis for Oracle Linux 8 since its support life is extended.* + + +### Ubuntu 22 Docker Images + +5.1.0 has replaced the CentOS 7 docker images with Ubuntu 22-based images. In most cases this should be a drop-in replacement but see the [Docker documentation](https://docs.perfsonar.net/release_candidates/5.1.0/install_docker.html) for more details. + +### Installation Helper Script + +To assist with installation independent of operating system, perfSONAR is making available a helper script that detects operating systems, sets-up package repositories and installs the selected bundle. Examples that will install the 5.1.0 beta are shown below: + +``` +# perfSONAR Toolkit +curl -s https://raw.githubusercontent.com/perfsonar/project/installation-script/install-perfsonar | sh -s - --repo staging toolkit + +# perfSONAR Testpoint +curl -s https://raw.githubusercontent.com/perfsonar/project/installation-script/install-perfsonar | sh -s - --repo staging testpoint + +# perfSONAR Archive +curl -s https://raw.githubusercontent.com/perfsonar/project/installation-script/install-perfsonar | sh -s - --repo staging archive + +``` + + +Bug Fixes and Other Changes +--------------------- + +### Archive + +- Upgraded to OpenSearch 2.13.0. Note that for the Debian package you must now set the OPENSEARCH_INITIAL_ADMIN_PASSWORD environment variable on first install of the core, toolkit and archive bundles. Example: `env OPENSEARCH_INITIAL_ADMIN_PASSWORD=perfSONAR123! apt install perfsonar-archive`. This is a requirement of the official Opensearch Debian package and not something introduced by the perfSONAR team. The password set will be overwritten with a random password by the perfSONAR installation process, so you can copy/paste the command shown safely. This does not affect existing installations that are updating. Alternatively you can use the Installation Helper Script detailed earlier in these release notes. +- Fixed issue with default permissions where SQL queries were not allowed +- Multiple fixes to the initial Opensearch configuration process. + +### Elmond + +- Removed references to 127.0.0.1 for better IPv6-only support + +### Graphs + +- This component is now end-of-life and replaced by Grafana + +### Logstash + +- Updated logstash to 8.10.4 +- Add "prometheus-pipeline" for collecting host and application metrics +- Fixed issue with null reverse hostname lookups + +### LS Cache Daemon + +- This component is now end-of-life + +### LS Registration Daemon + +- Updated systemd unit file to support automatic restarts and simplify process forking +- Various SELinux updates + +### MaDDash + +- This component is now end-of-life and replaced by Grafana + +### Nagios + +- This component is now end-of-life as its primary purpose was in support for MaDDash + +### OWAMP/TWAMP + +- Updated systemd unit file to enable auto-restarts +- Various SELinux updates +- Removed references to 127.0.0.1 for better IPv6-only support +- Fixed issue with TWAMP default socket location + +### Perl Shared Libraries + +- Removed references to 127.0.0.1 for better IPv6-only support +- Removed unused perl dependencies + +### pScheduler + +- Fixed issue where slip could reduce to negative +- Fixed issue when some versions of ping would not accept a floating point deadline and improved error reporting +- Fixed issue with RTT where deadline was not incorporated into expected duration +- Add ip-version option to MTU test and tool +- Added source-bind and dest-bind options to throughput test +- Added --full switch to "task" and "watch" command for more diagnostics +- Add mss support to iperf2 +- Add last scheduled/finished to "troubleshoot" output +- Replaced references to JSON template with Jinja2 +- Better handling of broken pipes during archiving +- Added protocol parameter to latency and latencybg +- Added port option to simplestreamer +- Multiple improvement around tool selection and error reporting +- Multiple fixes for handling iperf2 output +- Removed references to 127.0.0.1 for better IPv6-only support +- Better handle output when traceroute reports no hops + +### pSTracerouteViewer + +- This component is now end-of-life and replaced by Grafana + +### Toolkit + +- Removed ServiceWatcher since auto-restarts now handled by systemd and other cleanup handled by individual packages. +- Numerous SELinux fixes to cleanup denied actions +- Removed references to 127.0.0.1 for better IPv6-only support +- Added symlink to pscheduler logs from /var/log/perfsonar on Debian to match existing RPM behavior + + +Raw changes +----------- +- [archive](https://github.com/perfsonar/archive/compare/v5.0.8...5.1.0) +- [bundles](https://github.com/perfsonar/bundles/compare/v5.0.8...5.1.0) +- [elmond](https://github.com/perfsonar/elmond/compare/v5.0.8...5.1.0) +- [i2util](https://github.com/perfsonar/i2util/compare/v5.0.8...5.1.0) +- [logstash](https://github.com/perfsonar/logstash/compare/v5.0.8...5.1.0) +- [ls-registration-daemon](https://github.com/perfsonar/ls-registration-daemon/compare/v5.0.8...5.1.0) +- [owamp](https://github.com/perfsonar/owamp/compare/v5.0.8...5.1.0) +- [perl-shared](https://github.com/perfsonar/perl-shared/compare/v5.0.8...5.1.0) +- [psconfig](https://github.com/perfsonar/psconfig/compare/v5.0.8...5.1.0) +- [pscheduler](https://github.com/perfsonar/pscheduler/compare/v5.0.8...5.1.0) +- [toolkit](https://github.com/perfsonar/toolkit/compare/v5.0.8...5.1.0) \ No newline at end of file