doc: add documentation on how to write integration tests

doc/sphinx/development/setup.rst

@@ -15,7 +15,7 @@ access the files, and potentially modify them (such as for ``perltidy``).
 
 The supported tools are as follows:
 
-.. code-block:: shell
+.. code-block:: none
    :emphasize-lines: 1
 
    Usage: ./docker/devenv/run-tool.sh COMMAND [OPTIONS]
@@ -47,21 +47,21 @@ is automatically checked by ``perlcritic``, ``perltidy`` and ``shellcheck`` each
 
 If you previously cloned the repository with such a command:
 
-.. code-block:: shell
+.. code-block:: none
    :emphasize-lines: 1
 
    git clone https://github.com/ovh/the-bastion
 
 Then you can copy the provided :file:`pre-commit` script into your local :file:`.git` folder:
 
-.. code-block:: shell
+.. code-block:: none
    :emphasize-lines: 1
 
    cp contrib/git/pre-commit .git/hooks/pre-commit
 
 To verify that it works checkout a new test branch and add two dummy files like this:
 
-.. code-block:: shell
+.. code-block:: none
    :emphasize-lines: 1-5
 
    git checkout -B mybranch
@@ -97,8 +97,8 @@ To verify that it works checkout a new test branch and add two dummy files like
 
 As you see, the checks are running before the commit is validated and abort it should any check fail.
 
-Integration tests run
-*********************
+Running integration tests
+*************************
 
 Using Docker
 ------------
@@ -110,7 +110,7 @@ and tests the return values against expected output.
 
 To test the current code, use the following script, which will run ``docker build`` and launch the tests:
 
-.. code-block:: shell
+.. code-block:: none
    :emphasize-lines: 1
 
    tests/functional/docker/docker_build_and_run_tests.sh <TARGET>
@@ -120,15 +120,15 @@ You'll get a list of the supported targets by calling the command without argume
 
 For example, if you want to test it under Debian (which is a good default OS if you don't have any preference):
 
-.. code-block:: shell
+.. code-block:: none
    :emphasize-lines: 1
 
    tests/functional/docker/docker_build_and_run_tests.sh debian12
 
 The full tests usually take 25 to 50 minutes to run, depending on your hardware specs.
 If you want to launch only a subset of the integration tests, you may specify it:
 
-.. code-block:: shell
+.. code-block:: none
    :emphasize-lines: 1
 
    tests/functional/docker/docker_build_and_run_tests.sh debian12 --module=320-base.sh

doc/sphinx/development/tests.rst

@@ -1,41 +1,180 @@
+Writing tests
 =============
-Running Tests
-=============
 
-Using Docker
-============
+.. contents::
+
+When modifying code, adding features or fixing bugs, you're expected to write one or more tests to ensure that
+the feature your adding works correctly, or that the bug you've fixed doesn't come back.
+
+Integration tests modules live in the :file:`tests/functional/tests.d` folder.
+You may either add a new file to test your feature, or modify an existing file.
+
+These modules are shell scripts, and are sourced by the main integration test engine. Having a look at one of
+these modules will help you understand how they work, the :file:`tests/functional/tests.d/320-base.sh` is a good
+example you might want to look at.
+
+Example
+-------
+
+Here is a simple test taken from :file:`320-base.sh`:
+
+.. code-block:: none
+   :caption: a simple test
+
+   success   help2     $a0 --osh help
+   contain "OSH help"
+   json .error_code OK .command help .value null
+
+A complete reference of such commands can be found below, but let's explain this example in a few words:
+
+The command ``success`` implies that we're running a new test command, and that we expect it to work (we might
+also want to test invalid commands and ensure they fail as they should).
+The tester docker will connect to the target docker (that is running the bastion code) as a bastion user, and
+run the ``--osh help`` command there. This is expected to exit with a code indicating success (0),
+otherwise this test fails.
+
+The output of the command, once run on the bastion, should contain the text ``OSH help``, or the test will fail.
+
+In the JSON output (see :doc:`/using/api`) of this command, we expect to find the ``error_code`` field set to ``OK``,
+the ``command`` field set to ``help``, and the ``value`` field set to ``null``, or the test will fail.
+
+Running just this test will yield the following output:
+
+.. code-block:: none
+   :caption: a simple test output
+
+   00m04 [--] *** [0010/0021] 320-base::help2 (timeout --foreground 30 ssh -F /tmp/bastiontest.pgoA5h/ssh_config -i /tmp/bastiontest.pgoA5h/account0key1file user.5000@bastion_debian10_target -p 22 -- --json-greppable --osh help)
+   00m05 [--] [ OK ] RETURN VALUE (0)
+   00m05 [--] [ OK ] MUST CONTAIN (OSH help)
+   00m05 [--] [ OK ] JSON VALUE (.error_code => OK) [  ]
+   00m05 [--] [ OK ] JSON VALUE (.command => help) [  ]
+   00m05 [--] [ OK ] JSON VALUE (.value => null) [  ]
+
+As you can see, this simple test actually checked 5 things: the return value, whether the output text contained
+a given string, and 3 fields of the JSON output.
+
+Reference
+---------
+
+These are functions that are defined by the integration test engine and should be used in the test modules.
+
+Launch a test
+*************
+
+run
++++
+
+.. admonition:: syntax
+   :class: cmdusage
+
+   - run <name> <command>
+
+This function runs a new test named ``<name>``, which will execute ``<command>`` on the tester docker.
+Usually ``<command>`` will connect to the target docker (running the bastion code) using one of the test accounts,
+and run a command there.
+
+A few accounts are preconfigured:
+
+- The main account ("account 0"): this one is guaranteed to always exist at all times, and is a bastion admin.
+  There are a few variables that can be referenced to use this account:
+
+  - ``$a0`` is the ssh command-line to connect to the remote bastion as this account
+  - ``$account0`` is the account name, to be used in parameters of ``--osh`` commands where needed
+
+- A few secondary accounts that are created, deleted, modified during the tests:
+
+  - ``$a1``, ``$a2`` and ``$a3`` are the ssh command-lines to connect to the remote bastion as these accounts
+  - ``$account1``, ``$account2`` and ``$account3`` are the accounts names
+
+- Another special non-bastion-account command exists:
+
+  - ``$r0`` is the required command-line to directly connect to the remote docker on which the bastion code is running,
+    as root, with a bash shell. Only use this to modify the remote bastion files, such as config files, between tests
+
+A few examples follow:
+
+.. code-block:: none
+   :caption: running a few test commands
+
+   run test1 $a0 --osh info
+   run test2 $a0 --osh accountInfo --account $account1
+   run test3 $a1 --osh accountDelete --account $account2
+
+Note that the ``run`` function just runs the given command, but doesn't check whether it exited normally, you'll
+need other functions to verify this, see below.
+
+success
++++++++
+
+.. admonition:: syntax
+   :class: cmdusage
+
+   - success <name> <command>
+
+This function is exactly the same as the ``run`` command above, except that it expects the given ``<command>`` to
+return a valid error code (zero). Most of the time, you should be using this instead of ``run``, except if you're
+expecting the command to fail, in which case you should use ``run`` + ``retvalshouldbe``, see below.
+
+plgfail
++++++++
+
+.. admonition:: syntax
+   :class: cmdusage
+
+   - plgfail <name> <command>
+
+This function is exactly the same as the ``run`` command above, except that it expects the given ``<command>`` to
+return an error code of 100, which is the standard exit value when an osh command fails.
+
+This function is equivalent to using ``run`` followed by ``retvalshouldbe 100`` (see below).
+
+Verify a test validity
+**********************
+
+retvalshouldbe
+++++++++++++++
+
+.. admonition:: syntax
+   :class: cmdusage
+
+   - retvalshouldbe <value>
+
+Verify that the return value of a test launched right before with the ``run`` function is ``<value>``.
+You should use this if you expect the previous test to return a non-zero value.
 
-Functional tests use ``Docker`` to spawn an environment matching a bastion install.
+Note that the ``success`` function is equivalent to using ``run`` followed by ``retvalshouldbe 0``.
 
-One of the docker instances will be used as client, which will connect to the other instance
-which is used as the bastion server.
+contain
++++++++
 
-The client instance sends commands to the server instance and tests the return values against expected output.
+.. admonition:: syntax
+   :class: cmdusage
 
-To test the current code, put it on a machine with docker installed, and use the following script,
-which will run docker build and launch the tests:
+   - contain <text>
+   - contain REGEX <regex>
 
-    ``tests/functional/docker/docker_build_and_run_tests.sh <TARGET>``
+This function verifies that the output of the test contains a given ``<text>``. If you need to use a regex
+to match the output, you can use the ``contain REGEX`` construction, followed by the regex.
 
-Where target is one of the supported OSes. Currently only Linux targets are supported.
-You'll get a list of the supported targets by calling the command without argument.
+nocontain
++++++++++
 
-Without Docker
-==============
+.. admonition:: syntax
+   :class: cmdusage
 
-You can however still test the code against a BSD (or any other OS) without using Docker,
-by spawning a server under the target OS, and installing the bastion on it.
+   - nocontain <text>
+   - nocontain REGEX <regex>
 
-Then, from another machine, run:
+This function does the exact opposite of the ``contain`` function just above, and ensure that a given text
+or regex is NOT present in the output.
 
-    ``test/functional/launch_tests_on_instance.sh <IP> <port> <remote_user_name> <ssh_key_path> [outdir]``
+json
+++++
 
-Where ``IP`` and ``port`` are the information needed to connect to the remote server to test,
-``remote_user_name`` is the name of the account created on the remote bastion to use for the tests,
-and ``ssh_key_path`` is the private SSH key path used to connect to the account.
-The ``outdir`` parameter is optional, if you want to keep the raw output of each test.
+.. admonition:: syntax
+   :class: cmdusage
 
-This script is also the script used by the Docker client instance,
-so you're sure to get the proper results even without using Docker.
+   - json <field1> <value1> [<field2> <value2> ...]
 
-Please do **NOT** run any of those tests on a production bastion!
+This function checks the JSON API output of the test, and validates that it contains the correct value for each
+specified field. The ``<fieldX>`` entries must be valid `jq` filters.

doc/sphinx/index.rst

@@ -35,7 +35,9 @@ The third section focuses on the **USAGE** of the bastion, from the perspective
 
 The fourth section is about the proper **ADMINISTRATION** of the bastion itself. If you're about to be the person in charge of managing the bastion for your company, you want to read that one carefully!
 
-The fifth section is the complete reference of all the **PLUGINS** that are the commands used to interact with the bastion accounts, groups, accesses, credentials, and more.
+The fifth section is about **DEVELOPMENT** and how to write code for the bastion. If you'd like to contribute, this is the section to read!
+
+The sixth section is the complete reference of all the **PLUGINS** that are the commands used to interact with the bastion accounts, groups, accesses, credentials, and more.
 
 The unavoidable and iconic FAQ is also available under the **PRESENTATION** section.
 
@@ -82,6 +84,7 @@ The unavoidable and iconic FAQ is also available under the **PRESENTATION** sect
    :caption: Development
 
    development/setup
+   development/tests
 
 .. _plugins: