Merge branch 'develop'

elabit · Sep 29, 2021 · 1e09130 · 1e09130
2 parents 93a6f4c + cb8f430
commit 1e09130
Show file tree

Hide file tree

Showing 11 changed files with 100 additions and 92 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -5,6 +5,11 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## 1.2.1.1 - 2021-09-29
+
+### Changed
+
+* Improved the documentation for HTML log integration WATO discovery rule
 
 ## 1.2 - 2021-09-27
 

diff --git a/README.md b/README.md
@@ -11,23 +11,24 @@
 ![desc](img/robotmk_banner.png)
 
 <!--ts-->
-- [Robotmk](#robotmk)
-  - [Description](#description)
-  - [State of development](#state-of-development)
-  - [Key features/components](#key-featurescomponents)
-  - [Usage scenarios](#usage-scenarios)
-  - [Requirements](#requirements)
-  - [Installation](#installation)
-  - [Documentation](#documentation)
-    - [`robotmk.yml` explained](#robotmkyml-explained)
-  - [Next developments](#next-developments)
-  - [Contributing](#contributing)
-  - [License](#license)
-  - [Credits/Thanks](#creditsthanks)
-    - [Supporters](#supporters)
-  - [Contributors ✨](#contributors-)
-
-<!-- Added by: runner, at: Sun Jul 25 08:41:08 UTC 2021 -->
+* [Robotmk](#robotmk)
+   * [Description](#description)
+   * [Key features/components](#key-featurescomponents)
+   * [Usage scenarios](#usage-scenarios)
+   * [State of development](#state-of-development)
+   * [Requirements](#requirements)
+   * [Installation](#installation)
+   * [Documentation](#documentation)
+   * [Learning Resources](#learning-resources)
+   * [Professional Support](#professional-support)
+   * [Next developments](#next-developments)
+   * [Contributing](#contributing)
+   * [License](#license)
+   * [Credits/Thanks](#creditsthanks)
+      * [Supporters](#supporters)
+   * [Contributors <g-emoji class="g-emoji" alias="sparkles" fallback-src="https://github.githubassets.com/images/icons/emoji/unicode/2728.png">✨</g-emoji>](#contributors-)
+
+<!-- Added by: runner, at: Mon Sep 27 05:07:09 UTC 2021 -->
 
 <!--te-->
 
@@ -41,30 +42,25 @@
 * [Checkmk](https://checkmk.com) is a state-of-the-art IT infrastructure monitoring system. 
 * **Robotmk** integrates the results of Robot Framework into Checkmk. It bridges the gap between infrastructure and application testing. 
 
-**Why do I need Robotmk?** 
+**The purpose of Robotmk** 
+
+Business applications have to be **available, performant** and **functioning**. 
 
 A monitoring system like Checkmk does a very good job to monitor your business' IT infrastructure with checks for Servers, Network devices, etc. 
 
 But in the end - the reason why you are running IT is to *provide a service* to users. 
 
-Therefore you shouldn't only monitor infrastructure, but also *the services*. And most important: do it like they do. Use a real browser, mouse and keyboard strokes. Test from End (the user) to End (your IT infrastructure as a whole). This is called **"End2End"-Testing**.
-
-Robot Framework can automate End2End-Tests for you (and much more). Integrating those tests into Checkmk is a great supplement.
-
-**Robotmk** acts as a bridge between Robot Framework and Checkmk. 
-
-## State of development
-
-**Is Robotmk stable? Can it be used in production?**
+Therefore you shouldn't only monitor infrastructure, but also *the services*. And most important: do it like *they* do: Use a real browser, mouse and keyboard strokes. Test from End (the user) to End (your IT infrastructure as a whole). This is called **"End2End"-Testing**.
 
-Fortunately, the development of Robotmk is driven by customers who believe in the project and use it already in their daily business. This is where worthful feedback and feature requests come from. 
-
-As bugs are getting solved and new features are coming in, there is no guarantee that after installing a new version of Robotmk settings, output formats etc. will be the same or at least compatible with the previous version. We try to communicate this in the [CHANGELOG](./CHANGELOG.md) as detailled as possible. 
-
-Incompatibilities will always be reflected in a major version change. As soon as the major version number is not changing, chances are good that all existing CMK rules for Robotmk will work.  
+**Robot Framework** can automate End2End-Tests for you (and much more). 
+Integrating those tests into **Checkmk** is a great supplement.
+**Robotmk** acts as a bridge between **Robot Framework** and **Checkmk**. 
 
 ## Key features/components
 
+The Robotmk project consists of different components working together: 
+
+* Robotmk **WATO pages** for 100% configuration via the checkmk web UI
 * Robotmk **bakery rule** - configures E2E clients:
   * Use the Checkmk WATO rule editor to decide which remote hosts should be deployed with the Robotmk plugin.
   * Define which suites should be executed on the test host
@@ -81,6 +77,13 @@ Incompatibilities will always be reflected in a major version change. As soon as
     * performance data: get graphs for any runtime. Even insidious performance changes can thus be detected.
     * service discovery level: rule-based splitting of Robot Framework results into different Checkmk services ("checks" in Checkmk) - without splitting the robot test. 
     * reduction of the output to the essential needs for an optimum result.
+    * (new) log integration: The check extracts the Robot HTML logs and integrates 
+      them into the Checkmk web UI: 
+
+![desc](img/robotmk_html_logs_animated.gif)
+
+*The integration of Robot HTML logs into Checkmk makes it easy to understand why an E2E test failed*
+
 * [Robotmk Keyword Library](https://pypi.org/project/robotframework-robotmk/):
   * Set custom monitoring states
   * Add monitoring metrics
@@ -96,6 +99,19 @@ Read the [feature page](https://robotmk.org) of Robotmk to learn about its histo
 * monitoring modern apps: Angular, React, Android/iOS based, ... Robot Framework has a long list of well-curated libraries
 * monitoring old legacy apps: even the oldest applications can be monitored with Robot Framework by using a image recognition based library. 
 * monitoring 3rd party services: there are bunch of libraries to write tests based on REST, SOAP, TCP sockets, SSH, FTP, ...  
+* **Monitoring business relevant services** to save time, cost and revenue. 
+
+## State of development
+
+**Is Robotmk stable? Can it be used in production?**
+
+Fortunately, the development of Robotmk is driven by customers who believe in the project and use it already in their daily business. This is where worthful feedback and feature requests come from. 
+
+As bugs are getting solved and new features are coming in, there is no guarantee that after installing a new version of Robotmk settings, output formats etc. will be the same or at least compatible with the previous version. We try to communicate this in the [CHANGELOG](./CHANGELOG.md) as detailled as possible. 
+
+Incompatibilities will always be reflected in a major version change. As soon as the major version number is not changing, chances are good that all existing CMK rules for Robotmk will work.  
+
+Remark to the current version 1.2: although this version will break existing WATO rules for a very last time (see #164) this is not a major release.
 
 ## Requirements
 
@@ -134,58 +150,41 @@ The context help can be shown by clicking on the **book icon** in the top right
 
 ![How to show the context help](img/show_context_help.gif)
 
-### `robotmk.yml` explained
+## Learning Resources
 
-**Warning:** The format of `robotmk.yml` can change at any time! This example may be outdated.
+You can find comprehensive learning material to teach you the best practices in End2End monitoring with Checkmk and Robotmk on [the Robotmk Academy](https://www.robotmk.org).
 
-This section is important if you decide not to use the CMK Enterprise edition (CEE) and to generate the `robotmk.yml` file instead by hand or automated by Ansible, Salt etc. 
+Registration is free and includes three basic course which show the fundamentals of Robotmk. 
 
-Example of `robotmk.yml`:
-
-```
-global:
-  agent_output_encoding: zlib_codec
-  cache_time: 960
-  execution_interval: 900
-  execution_mode: agent_serial
-  log_rotation: '14'
-  logging: true
-  robotdir: /usr/lib/check_mk_agent/robot
-  transmit_html: true
-suites: {}
-```
+Further learning course material is in preparation for 
 
-Explanation: 
+* Containerized E2E testing with Docker Swarm/Kubernetes/OpenShift
+* Best practices ofr using libraries like SeleniumLibrary, Playwright, CryptoLibrary etc.
+* fully automated Python environment rollouts with RCC
+* Using Robocorp with Robotmk 
+* and much more. 
 
-* `global:`
-  * `agent_output_encoding`: `zlib_codec|utf_8|base64_codec` Choose `zlib_codec`; the other options are for debugging purposes. Zlib guarantees bandwidth saving. 
-  * `cache_time`
-  * `execution_interval` (only in serial mode) 
-  * `execution_mode`: `agent_serial|external`
-  * `log_rotation`: Number of days to rotate the logs
-  * `logging`: `true|false`
-  * `robotdir`: The folder containing `.robot` files and/or suite dirs 
-  * `transmit_html`: `true|false` - whether to transmit the Robot HTML log to the CMK server. 
-* `suites:` - if Robotmk should execute everythin it can find in `robotdir`, set this to `{}` (empty dict)
-  * `suiteid`: either equal to `path` (see below) or, if the same suite must be executed multiple times, a combination of `path` and `tag`
-    * `path`: relative path to the `.robot` file or a folder within the `robotdir` (the latter one is highly recommended)
-    * `piggybackhost` (optional): if set, the result of this suite will be assigned to another CMK host.
-    * `robot_params` (optional): a list of Robot framework command line switches 
+## Professional Support
 
+Need the help of an Expert? Save your time.
 
+No matter at which stage you are with Robot Framework and Robotmk: there are [experienced professionals](https://www.robotmk.org/pages/professionalsupport) who can assist you with: 
 
+* Robotmk best practice for the optimal usage of Robot Framework together with Checkmk * monitoring
+* Implementation
+* Know-how: all partners are up to date with recent developments and features
+* Deep knowledge of configuration tools like Saltstack, Ansible etc. to implement a fully * automated deployment of Robotmk, Robot Framework, Libraries and test code.
+* Checkmk best practice
 
 ## Next developments
 
 See the [Github Issues](https://github.com/simonmeggle/robotmk/issues) page for a complete list of feature requests, known bugs etc.
 
 Next development steps will be: 
 
-* It is helpful to have also Robot Logs at hand when there is an alarm. It is planned
-  that the Robotmk plugin also collects the Robot HTML logs and transfers them to the
-  CMK server. HTML logs could also include (embedded?) screenshots, animated GIF screen-recordings etc. 
-  The goal is to implement a service action button which guides you directly to the most
-  recent Robot Log. Read more about this idea in [issue #1](https://github.com/simonmeggle/robotmk/issues/1).   
+* Development of a Robotmk Docker container
+* Implementation of a Robotmk special agent for Kubernetes/Docker Swarm environments  
+* Implementation of a Robotmk special agent for Robocorp Cloud
 * Integrate RCC from the [Robocorp toolchain](https://robocorp.com/docs/rcc/overview) to make Python environments portable. 
 
 ## Contributing

diff --git a/agents_plugins/robotmk.py b/agents_plugins/robotmk.py
@@ -49,7 +49,7 @@
 
 local_tz = datetime.utcnow().astimezone().tzinfo
 
-ROBOTMK_VERSION = 'v1.2'
+ROBOTMK_VERSION = 'v1.2.1.1'
 
 class RMKConfig():
     _PRESERVED_WORDS = [

diff --git a/bakery/v1/robotmk.py b/bakery/v1/robotmk.py
@@ -18,7 +18,7 @@
 # to the Free Software Foundation, Inc., 51 Franklin St,  Fifth Floor,
 # Boston, MA 02110-1301 USA.
 
-ROBOTMK_VERSION = 'v1.2'
+ROBOTMK_VERSION = 'v1.2.1.1'
 
 import cmk.utils.paths
 import os

diff --git a/bakery/v2/robotmk.py b/bakery/v2/robotmk.py
@@ -18,7 +18,7 @@
 # to the Free Software Foundation, Inc., 51 Franklin St,  Fifth Floor,
 # Boston, MA 02110-1301 USA.
 
-ROBOTMK_VERSION = 'v1.2'
+ROBOTMK_VERSION = 'v1.2.1.1'
 
 from typing import Iterable, TypedDict, List
 from pathlib import Path

diff --git a/checks/v1/robotmk b/checks/v1/robotmk
@@ -34,7 +34,7 @@ from collections import namedtuple
 iam = "robotmk"
 # DO NOT DELETE
 inventory_robotmk_rules = []
-ROBOTMK_VERSION = 'v1.2'
+ROBOTMK_VERSION = 'v1.2.1.1'
 DEFAULT_SVC_PREFIX = 'Robot Framework E2E $SUITEID$SPACE-$SPACE'
 HTML_LOG_DIR = "%s/%s" % (os.environ['OMD_ROOT'], 'local/share/addons/robotmk')
 

diff --git a/checks/v2/robotmk.py b/checks/v2/robotmk.py
@@ -35,7 +35,7 @@
 from cmk.base.plugins.agent_based.agent_based_api.v1 import *
 from cmk.utils.exceptions import MKGeneralException
 
-ROBOTMK_VERSION = 'v1.2'
+ROBOTMK_VERSION = 'v1.2.1.1'
 DEFAULT_SVC_PREFIX = 'Robot Framework E2E $SUITEID$SPACE-$SPACE'
 HTML_LOG_DIR = "%s/%s" % (os.environ['OMD_ROOT'], 'local/share/addons/robotmk')
 

diff --git a/gh-md-toc b/gh-md-toc
@@ -23,7 +23,7 @@
 #       substr($0, match($0, "href=\"[^\"]+?\" ")+6, RLENGTH-8)
 #
 
-gh_toc_version="0.7.0"
+gh_toc_version="0.8.0"
 
 gh_user_agent="gh-md-toc v$gh_toc_version"
 

diff --git a/img/robotmk_html_logs_animated.gif b/img/robotmk_html_logs_animated.gif
diff --git a/release.sh b/release.sh
@@ -62,6 +62,7 @@ function release() {
     header "Pushing ..."
     git push origin master
     git push origin $VTAG
+    git checkout develop
 }
 
 function unrelease() {

diff --git a/web_plugins/wato/robotmk_wato_params_discovery.py b/web_plugins/wato/robotmk_wato_params_discovery.py
@@ -58,25 +58,28 @@
 
 inventory_dict_robotmk_htmllog = (
     Tuple(
-        title=_("Restrict the <b>HTML log files</b> link creation"),
-        help=_("""If Robotmk is configured to <i>Transmit HTML logs to server</i>, it can then 
-        display two action icons right of each discovered service which allows to open the <b>last error log</b> and the <b>current log</b>.<br>
-        In order to make this work, just follow these two steps: <br><br>>
-        <b>1.</b> Add the following lines to <tt>$OMD_ROOT/etc/apache/conf.d/check_mk.conf.conf</tt> (replace <tt>$SITE</tt> accordingly):<br>
-        <tt>Alias /$SITE/check_mk/addons "/omd/sites/$SITE/local/share/addons"<br>
-        Directory "/omd/sites/$SITE/local/share/addons"<br>
+        title=_("<b>HTML log file</b> integration (experimental)"),
+        help=_("""Robotmk can display two action icons right of each discovered service which allows to open the <b>last error log</b> and the <b>current log</b>.<br>
+        In order to make this work, just follow these two steps: <br><br>
+        <b>1.</b> In rule <i>Robotmk (Linux, Windows)</i>, configure Robotmk clients to <i>Transmit HTML logs to server</i>.<br>
+        <b>2.</b> Add the following lines to <tt>$OMD_ROOT/etc/apache/conf.d/check_mk.conf</tt> (replace <tt>--SITE--</tt> with your site name):<br><br>
+        <tt>Alias /--SITE--/check_mk/addons "/omd/sites/--SITE--/local/share/addons"<br>
+        <Directory "/omd/sites/--SITE--/local/share/addons"><br>
         Options +Indexes<br>
         AllowOverride None<br>
-        /Directory</tt><br>
-        <b>2.</b> In <i>Global settings</i>, create <b>two custom actions</b> pointing to the URLs:<br>
-        * icon <i>robotmk80.png</i>: <tt>addons/robotmk/$HOSTNAME_URL_ENCODED$/$SERVICEDESC$/suite_last_log.html</tt><br>
-        * icon <i>robotmk80_dot.png</i>: <tt>addons/robotmk/$HOSTNAME_URL_ENCODED$/$SERVICEDESC$/suite_last_error_log.html</tt><br>
-        <b>3.</b> Create <b>two rules</b> to <b>assign the created actions</b>:<br>
-        * action id for <i>last log</i>: "service label is <tt>robotmk/html_last_log:yes</tt>"<br>
-        * action id for <i>last error log</i>: "service label is <tt>robotmk/html_last_error_log:yes</tt>"<br><br>  
-        The regular expressions here define where the service label will be shown. <br>Default = <tt>.*</tt> = show on all services.<br><br>
-
-        Remark: The host must exist with the real hostname/FQDN in Checkmk. Robotmk will save max. 2 HTML files per discovered suite to save hard disk space. However, it is advised to monitor the space usage on <tt>$OMD_ROOT/local/share/addons/robotmk</tt>."""),
+        </Directory></tt><br> <br>
+        <b>3.</b> In <i>Global settings > Custom icons and actions</i>, create <b>two custom actions</b>:<br>
+        * id:<tt>robotmk_last_log</tt>, tite: "Robot Framework: last HTML log", icon: <i>robotmk80.png</i>, action URL: <tt>addons/robotmk/$HOSTNAME_URL_ENCODED$/$SERVICEDESC$/suite_last_log.html</tt><br>
+        * id:<tt>robotmk_last_error_log</tt>, tite: "Robot Framework: last error HTML log", icon: <i>robotmk80_dot.png</i>, action URL: <tt>addons/robotmk/$HOSTNAME_URL_ENCODED$/$SERVICEDESC$/suite_last_error_log.html</tt><br>
+        <b>4.</b> In rule <i>Custom icons or actions for services in status GUI</i>, assign those actions with the following conditions:<br>
+        * action <tt>robotmk_last_log</tt>: condition "service label is <tt>robotmk/html_last_log:yes</tt>"<br>
+        * action <tt>robotmk_last_error_log</tt>: condition "service label is <tt>robotmk/html_last_error_log:yes</tt>"<br><br>
+
+        Remarks: <br>
+        - The host must exist with the real hostname/FQDN in Checkmk. Robotmk will save max. 2 HTML files per discovered suite to save hard disk space. However, it is advised to monitor the space usage on <tt>$OMD_ROOT/local/share/addons/robotmk</tt>.<br>
+        - Be advised that this feature is in an early beta state and changing the Apache configuration as above is not covered by customer support. Any authenticated user will be able to see <i>all</i> HTML logs.<br>
+        - HTML logs are always saved on the machine where the Robotmk check is running on; there is currently no way for distributed setups.<br><br>
+        The regular expressions below here define where the service label will be shown. <br>Default = <tt>.*</tt> = show HTML logs on all Robotmk services."""),
         show_titles=True,
         # orientation="horizontal",
         elements=[