Generic Sphinx-Needs JS test framework

.github/workflows/ci.yaml

@@ -28,6 +28,12 @@ jobs:
         uses: actions/setup-python@v4
         with:
           python-version: ${{ matrix.python-version }}
+      - name: Use Node.js
+        uses: actions/setup-node@v3
+        with:
+          node-version: 18
+      - name: Install Cypress Test Framework
+        run: npm install cypress
       - name: Install Nox Dependencies
         run: |
           python -m pip install poetry nox nox-poetry pyparsing==3.0.4

.gitignore

@@ -30,7 +30,8 @@ docs/_images/need_bar_*.*
 .dockerignore
 
 .benchmarks/
-
+tests/cypress/*
+tests/cypress/screenshots
 
 need_bar_*.png
 need_pie_*.png

AUTHORS

@@ -37,3 +37,5 @@ Duodu Randy <[email protected]>
 Christian Wappler <[email protected]>
 
 Chris Sewell <[email protected]>
+
+Simon Leiner <[email protected]>

Makefile

@@ -10,11 +10,15 @@ lint:
 
 .PHONY: test
 test:
-	poetry run pytest -n auto --tb=long --ignore=tests/benchmarks tests/
+	poetry run pytest -n auto -m "not jstest" --tb=long --ignore=tests/benchmarks tests/
 
 .PHONY: test
 test-short:
-	poetry run pytest -n auto --tb=long --ignore-glob="*official*" --ignore=tests/benchmarks tests/
+	poetry run pytest -n auto -m "not jstest" --tb=long --ignore-glob="*official*" --ignore=tests/benchmarks tests/
+
+.PHONY: test
+test-js:
+	poetry run pytest -n auto -m "jstest" --tb=long --ignore-glob="*official*" --ignore=tests/benchmarks tests/
 
 .PHONY: benchmark-time
 benchmark-time:

docs/_templates/layout.html

@@ -1,11 +1,6 @@
 {# Import the theme's layout. #}
 {% extends '!layout.html' %}
 
-{%- block extrahead %}
-    <script src="https://ajax.googleapis.com/ajax/libs/jquery/3.6.0/jquery.min.js"></script>
-    {{ super() }}
-{%- endblock %}
-
 {%- block footer %}
     {{ super() }}
 

docs/configuration.rst

@@ -1939,6 +1939,22 @@ constraints_passed is a bool showing if ALL constraints of a corresponding need
 
 constraints_results is a dictionary similar in structure to needs_constraints above. Instead of executable python statements, inner values contain a bool describing if check_0, check_1 ... passed successfully.
 
+.. versionadded:: 1.4.0
+
+    The ``"error_message"`` key can contain a string, with Jinja templating, which will be displayed if the constraint fails, and saved on the need as ``constraints_error``:
+
+    .. code-block:: python
+
+        needs_constraints = {
+
+            "critical": {
+                "check_0": "'critical' in tags",
+                "severity": "CRITICAL",
+                "error_message": "need {% raw %}{{id}}{% endraw %} does not fulfill CRITICAL constraint, because tags are {% raw %}{{tags}}{% endraw %}"
+            }
+
+        }
+
 
 .. code-block:: rst
 

docs/contributing.rst

@@ -121,6 +121,98 @@ These snapshots can be updated by running:
 
       pip install -r docs/requirements.txt
 
+Running JS Testcases with PyTest
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+**Setup Cypress Locally**
+
+* Install Node JS on your computer and ensure it can be accessed through the CMD.
+* Install Cypress using the npm package manager by running ``npm install cypress``. Visit this link for more information on `how to install Cypress <https://docs.cypress.io/guides/getting-started/installing-cypress#npm-install>`_.
+* Verify if Cypress is installed correctly and is executable by running: ``npx cypress verify``. Get out this page for more information about `Cypress commandline <https://docs.cypress.io/guides/guides/command-line>`_.
+* If everything is successful then we can use Cypress.
+
+**Enable Cypress Test in Python Test Files**
+
+* Under the ``js_test`` folder, you can save your Cypress JS test files (files should end with: ``*.cy.js``). For each Cypress JS test file, you will need to write the Cypress JS test cases in the file. You can read more from the `Cypress Docs <https://docs.cypress.io/>`_. You can also check the ``tests/js_test/sn-collapse-button.cy.js`` file as reference.
+* In your Python test files, you must mark every JS related test case with the marker - ``jstest`` and you must also set the ``spec_pattern`` key-value pair as part of the ``test_app`` fixture parameter. For example, your test case could look like this:
+    .. code-block:: python
+
+        # tests/test_sn_collapse_button.py
+
+        import pytest
+
+
+        @pytest.mark.jstest
+        @pytest.mark.parametrize(
+            "test_app",
+            [
+                {
+                    "buildername": "html",
+                    "srcdir": "doc_test/variant_doc",
+                    "tags": ["tag_a"],
+                    "spec_pattern": "js_test/js-test-sn-collapse-button.cy.js"
+                }
+            ],
+            indirect=True,
+        )
+        def test_collapse_button_in_docs(test_app):
+            ...
+
+    .. note::
+
+        The ``spec_pattern`` key is required to ensure Cypress locates your test files or folder. Visit this link for more info on how to set the `spec_pattern <https://docs.cypress.io/guides/guides/command-line#cypress-run-spec-lt-spec-gt>`_.
+
+* After you set the ``spec_pattern`` key-value pair as part of the ``test_app`` fixture parameter, you can call ``app.test_js()`` in your Python test case to run a JS test for the ``spec_pattern`` you provided. For example, you can use ``app.test_js()`` like below:
+    .. code-block:: python
+
+        # tests/test_sn_collapse_button.py
+
+        import pytest
+
+
+        @pytest.mark.jstest
+        @pytest.mark.parametrize(
+            "test_app",
+            [
+                {
+                    "buildername": "html",
+                    "srcdir": "doc_test/variant_doc",
+                    "tags": ["tag_a"],
+                    "spec_pattern": "js_test/js-test-sn-collapse-button.cy.js"
+                }
+            ],
+            indirect=True,
+        )
+        def test_collapse_button_in_docs(test_app):
+            """Check if the Sphinx-Needs collapse button works in the provided documentation source."""
+            app = test_app
+            app.build()
+
+            # Call `app.test_js()` to run the JS test for a particular specPattern
+            js_test_result = app.test_js()
+
+            # Check the return code and stdout
+            assert js_test_result["returncode"] == 0
+            assert "All specs passed!" in js_test_result["stdout"].decode("utf-8")
+
+    .. note::
+
+        ``app.test_js()`` will return a dictionary object containing the ``returncode``, ``stdout``, and ``stderr``. Example:
+
+        .. code-block:: python
+
+            return {
+                "returncode": 0,
+                "stdout": "Test passed string",
+                "stderr": "Errors encountered,
+            }
+
+* You can run the ``make test-js`` command to check all JS testcases.
+    .. note::
+
+        The ``http_server`` process invoked by the ``make test-js`` command may not terminate properly in some instances.
+        Kindly check your system's monitoring app to end the process if not terminated automatically.
+
 Linting & Formatting
 --------------------
 
@@ -257,7 +349,7 @@ The following is an outline of the build events which this extension adds to the
    - Check for dead links (``process_need_nodes -> check_links``)
    - Generate back links (``process_need_nodes -> create_back_links``)
    - Process constraints, for each ``Need`` node (``process_need_nodes -> process_constraints``)
-   - Perform all modifications on need data items, due to ``Needextend`` nodes (``process_need_nodes -> process_needextend``)
+   - Perform all modifications on need data items, due to ``Needextend`` nodes (``process_need_nodes -> extend_needs_data``)
    - Format each ``Need`` node to give the desired visual output (``process_need_nodes -> print_need_nodes``)
    - Process all other need specific nodes, replacing them with the desired visual output (``process_creator``)
 

docs/requirements.txt

@@ -7,4 +7,4 @@ sphinxcontrib-programoutput
 sphinx-design
 click
 tabulate
-sphinx-immaterial==0.11.7
+sphinx-immaterial==0.11.7