feat: option to skip including events into extra_vars (#716)

When we unconditionally send event data to the controller for a workflow
template the launch of the workflow template fails. With this option in
a rulebook the user can prevent event data from being sent by setting

   - include_events: false

The default value is true so existing rulebooks will keep sending events
to controller.
Fixes #622 
https://issues.redhat.com/browse/AAP-13456

Root Cause Analysis:
In the Controller if you are running a workflow template you cannot send
event data to it unconditionally like we can do with job template. We
get an error from Controller

**Check the Prompt on Launch setting on the Workflow Job Template to
include Extra Variables.**

If a survey spec is involved then there are more constraints if there
are certain parameters that are marked as required in the survey spec.

So we get 2 distinct items

-   Prompt on Launch for Extra Variables
-   Survey Spec

The following use cases are supported for workflow job template with and
without survey spec.

The use cases are based on a rulebook that looks like the following with
the action being updated for every case.
```
---
- name: Test run job templates
  hosts: all
  sources:
    - ansible.eda.generic:
         payload:
           - name: Fred
             age: 25
  rules:
    - name: "Run job template"
      condition: event.age == 25
      action:
        run_workflow_template:
          name: workflow_mk
          organization: Default
```

### **Case 1:**
If your Workflow Template doesn't support Prompt on Launch for Extra
Vars and a Survey Spec is not enabled

```
run_workflow_template:
          name: workflow_mk
          include_events: false
          organization: Default
```
### **Case 2:**
If your Workflow Template supports Prompt on Launch for Extra Vars and a
Survey Spec is not enabled

```
 run_workflow_template:
          name: workflow_mk
          organization: Default
          job_args:
            extra_vars:
              name: "{{ event.name }}"
```
The payload sent to Controller

```
name: Fred
ansible_eda:
  ruleset: Test run job templates
  rule: Run job template
  event:
    meta:
      source:
        name: ansible.eda.generic
        type: ansible.eda.generic
      received_at: '2024-09-26T05:29:10.983346Z'
      uuid: 9570c7f8-9bc9-438f-9199-803707e31ccf
    name: Fred
    age: 25
```
### **Case 3:**
If your Workflow Template doesn't support Prompt on Launch for Extra
Vars and a Survey Spec is enabled
```
 run_workflow_template:
          name: workflow_mk
          include_events: false
          organization: Default
          job_args:
            extra_vars:
              name: "{{ event.name }}"
```
The payload sent to Controller

```
 name: Fred
```

### **Case 4:**
If your Workflow Template supports Prompt on Launch for Extra Vars and a
Survey Spec is enabled
```
action:
        run_workflow_template:
          name: workflow_mk
          organization: Default
          job_args:
            extra_vars:
              name: "{{ event.name }}"
```

The payload sent to Controller
```
name: Fred
ansible_eda:
  ruleset: Test run job templates
  rule: Run job template
  event:
    meta:
      source:
        name: ansible.eda.generic
        type: ansible.eda.generic
      received_at: '2024-09-26T07:26:42.160723Z'
      uuid: 46169e51-ba6f-4a55-963b-b626fff7cfdc
    name: Fred
    age: 25
```

### **Case 5:** Survey Spec has a required parameter and its not
provided
```
action:
        run_workflow_template:
          name: workflow_mk
          organization: Default
```

Missing args
2024-09-26 12:59:03,738 - ansible_rulebook.job_template_runner - ERROR -
Error {'variables_needed_to_start': ["'name' value missing"]}

### **Case 6:** Survey Spec has no required parameter and its not
provided
```
action:
        run_workflow_template:
          name: workflow_mk
          organization: Default
```

The Default Value is provided by the Survey Spec and it works
```
name: fred
ansible_eda:
  ruleset: Test run job templates
  rule: Run job template
  event:
    meta:
      source:
        name: ansible.eda.generic
        type: ansible.eda.generic
      received_at: '2024-09-26T07:55:15.320036Z'
      uuid: 68a6c25d-dcf5-4a1c-8eb2-7432376d2d88
    name: Fred
    age: 25
```

CHANGELOG.md

@@ -5,6 +5,7 @@
 ### Changed
 ### Added
 ### Fixed
+- Allow user to optionally include matching events
 
 
 ## [1.1.1] - 2024-09-19

ansible_rulebook/action/helper.py

@@ -125,15 +125,23 @@ def embellish_internal_event(self, event: Dict) -> Dict:
     def set_action(self, action) -> None:
         self.action = action
 
-    def collect_extra_vars(self, user_extra_vars: Dict) -> Dict:
+    def collect_extra_vars(
+        self, user_extra_vars: Dict, include_events: bool = True
+    ) -> Dict:
         """When we send information to ansible-playbook or job template
-        on AWX, we need the rule and event specific information to
+        on AWX, we need the rule and optionally event specific information to
         be sent to this external process
 
         the caller passes in the user_extra_vars from the action args
         and then we append eda specific vars and return that as a
         the updated dictionary that is sent to the external process
+
+        if the caller doesn't want to include events data return the
+        user_extra_vars.
         """
+        if not include_events:
+            return user_extra_vars
+
         extra_vars = user_extra_vars.copy() if user_extra_vars else {}
 
         eda_vars = {

ansible_rulebook/action/run_job_template.py

@@ -72,8 +72,10 @@ async def __call__(self):
         )
 
         self.job_args["extra_vars"] = self.helper.collect_extra_vars(
-            self.job_args.get("extra_vars", {})
+            self.job_args.get("extra_vars", {}),
+            self.action_args.get("include_events", True),
         )
+
         await self._job_start_event()
         await self._run()
 

ansible_rulebook/action/run_workflow_template.py

@@ -70,10 +70,11 @@ async def __call__(self):
             self.helper.metadata.rule_set,
             self.helper.metadata.rule,
         )
-
         self.job_args["extra_vars"] = self.helper.collect_extra_vars(
-            self.job_args.get("extra_vars", {})
+            self.job_args.get("extra_vars", {}),
+            self.action_args.get("include_events", True),
         )
+
         await self._job_start_event()
         await self._run()
 

ansible_rulebook/job_template_runner.py

@@ -209,13 +209,6 @@ async def run_workflow_job_template(
                 "Workflow template %s does not accept limit, removing it", name
             )
             job_params.pop("limit")
-        if not obj["ask_variables_on_launch"] and "extra_vars" in job_params:
-            logger.warning(
-                "Workflow template %s does not accept extra vars, "
-                "removing it",
-                name,
-            )
-            job_params.pop("extra_vars")
         job = await self._launch(job_params, url)
         return await self._monitor_job(job["url"])
 

ansible_rulebook/schema/ruleset_schema.json

@@ -453,6 +453,10 @@
                         },
                         "delay": {
                             "type": "integer"
+                        },
+                        "include_events": {
+                            "type": "boolean",
+                            "default": true
                         }
                     },
                     "required": [
@@ -502,6 +506,10 @@
                         },
                         "delay": {
                             "type": "integer"
+                        },
+                        "include_events": {
+                            "type": "boolean",
+                            "default": true
                         }
                     },
                     "required": [

docs/actions.rst

@@ -147,6 +147,9 @@ Run a job template.
    * - organization
      - The name of the organization
      - Yes
+   * - include_events
+     - Should we include the matching events in the payload sent to controller. Default is true
+     - No
    * - set_facts
      - The artifacts from the job template execution are inserted back into the rule set as facts
      - No
@@ -203,6 +206,9 @@ Run a workflow template.
    * - organization
      - The name of the organization
      - Yes
+   * - include_events
+     - Should we include the matching events in the payload sent to controller. Default is true. If your workflow template does not have Prompt on Launch for Extra Variables or a Survey spec, you will have to set this to false.
+     - No
    * - set_facts
      - The artifacts from the workflow template execution are inserted back into the rule set as facts
      - No

sonar-project.properties

@@ -0,0 +1,29 @@
+# Complete documentation with many more options at:
+# https://docs.sonarqube.org/latest/analysis/analysis-parameters/
+
+## The unique project identifier. This is mandatory.
+# Do not duplicate or reuse!
+# Available characters: [a-zA-Z0-9_:\.\-]
+# Must have least one non-digit.
+# Recommended format: <group>:<project>
+sonar.projectKey=ansible_ansible-rulebook
+
+sonar.organization=ansible
+
+# Customize what paths to scan. Default is .
+sonar.sources=.
+
+# Verbose name of project displayed in WUI. Default is set to the projectKey. This field is optional.
+sonar.projectName=ansible-rulebook
+
+sonar.issue.ignore.multicriteria=e1
+# Ignore "should be a variable"
+#sonar.issue.ignore.multicriteria.e1.ruleKey=python:S1192
+sonar.issue.ignore.multicriteria.e1.resourceKey=**/action/*
+
+# Only scan with python3
+sonar.python.version=3.9,3.10,3.11,3.12
+
+# Ignore code dupe for premature code reuse recommened in PR
+# https://github.com/ansible/ansible-rulebook/pull/537#issuecomment-1598883529
+sonar.cpd.exclusions=**/action/*

tests/examples/84_job_template_exclude_events.yml

@@ -0,0 +1,20 @@
+---
+- name: Test run job templates without event payload
+  hosts: all
+  sources:
+    - ansible.eda.generic:
+        payload:
+          - age: 55
+            name: Fred
+            zip: 12345
+  rules:
+    - name: "Run job template"
+      condition: event.name == "Fred"
+      action:
+        run_job_template:
+          name: Demo Job Template
+          organization: Default
+          include_events: false
+          job_args:
+            extra_vars:
+              name: "{{ event.name }}"

tests/examples/85_workflow_template_exclude_events.yml

@@ -0,0 +1,20 @@
+---
+- name: Test run workflow templates without event payload
+  hosts: all
+  sources:
+    - ansible.eda.generic:
+        payload:
+          - age: 55
+            name: Fred
+            zip: "12345"
+  rules:
+    - name: "Run workflow template"
+      condition: event.name == "Fred"
+      action:
+        run_workflow_template:
+          name: Demo Workflow Template
+          organization: Default
+          include_events: false
+          job_args:
+            extra_vars:
+              name: "{{ event.name }}"

tests/examples/86_job_template_include_events.yml

@@ -0,0 +1,19 @@
+---
+- name: Test run job templates with event payload
+  hosts: all
+  sources:
+    - ansible.eda.generic:
+        payload:
+          - age: 55
+            name: Fred
+            zip: 12345
+  rules:
+    - name: "Run job template"
+      condition: event.name == "Fred"
+      action:
+        run_job_template:
+          name: Demo Job Template
+          organization: Default
+          job_args:
+            extra_vars:
+              name: "{{ event.name }}"

tests/examples/87_workflow_template_include_events.yml

@@ -0,0 +1,19 @@
+---
+- name: Test run workflow templates with event payload
+  hosts: all
+  sources:
+    - ansible.eda.generic:
+        payload:
+          - age: 55
+            name: Fred
+            zip: "12345"
+  rules:
+    - name: "Run workflow template"
+      condition: event.name == "Fred"
+      action:
+        run_workflow_template:
+          name: Demo Workflow Template
+          organization: Default
+          job_args:
+            extra_vars:
+              name: "{{ event.name }}"

tests/examples/88_job_template_no_args.yml

@@ -0,0 +1,16 @@
+---
+- name: Test run job templates with no args
+  hosts: all
+  sources:
+    - ansible.eda.generic:
+        payload:
+          - age: 55
+            name: Fred
+            zip: 12345
+  rules:
+    - name: "Run job template"
+      condition: event.name == "Fred"
+      action:
+        run_job_template:
+          name: Demo Job Template
+          organization: Default

tests/test_examples.py

@@ -2434,3 +2434,97 @@ async def test_83_boolean_true():
             ],
         }
         await validate_events(event_log, **checks)
+
+
+INCLUDE_EVENTS_RUN = [
+    (
+        (
+            "ansible_rulebook.action.run_job_template."
+            "job_template_runner.run_job_template"
+        ),
+        "examples/84_job_template_exclude_events.yml",
+        "run_job_template",
+        "https://examples.com/#/jobs/945/details",
+        ["name"],
+    ),
+    (
+        (
+            "ansible_rulebook.action.run_workflow_template."
+            "job_template_runner.run_workflow_job_template"
+        ),
+        "examples/85_workflow_template_exclude_events.yml",
+        "run_workflow_template",
+        "https://examples.com/#/jobs/workflow/945/details",
+        ["name"],
+    ),
+    (
+        (
+            "ansible_rulebook.action.run_job_template."
+            "job_template_runner.run_job_template"
+        ),
+        "examples/86_job_template_include_events.yml",
+        "run_job_template",
+        "https://examples.com/#/jobs/945/details",
+        ["name", "ansible_eda"],
+    ),
+    (
+        (
+            "ansible_rulebook.action.run_workflow_template."
+            "job_template_runner.run_workflow_job_template"
+        ),
+        "examples/87_workflow_template_include_events.yml",
+        "run_workflow_template",
+        "https://examples.com/#/jobs/workflow/945/details",
+        ["name", "ansible_eda"],
+    ),
+    (
+        (
+            "ansible_rulebook.action.run_job_template."
+            "job_template_runner.run_job_template"
+        ),
+        "examples/88_job_template_no_args.yml",
+        "run_job_template",
+        "https://examples.com/#/jobs/945/details",
+        ["ansible_eda"],
+    ),
+]
+
+
+@pytest.mark.asyncio
+@pytest.mark.parametrize(
+    "location,rulebook,action_type,job_url,expected_keys", INCLUDE_EVENTS_RUN
+)
+async def test_include_events(
+    location, rulebook, action_type, job_url, expected_keys
+):
+    ruleset_queues, event_log = load_rulebook(rulebook)
+
+    queue = ruleset_queues[0][1]
+    rs = ruleset_queues[0][0]
+    response_obj = dict(
+        status="successful", id=945, created="dummy", artifacts=dict(a=1)
+    )
+    job_template_runner.host = "https://examples.com"
+    with SourceTask(rs.sources[0], "sources", {}, queue):
+        with patch(
+            location,
+            return_value=response_obj,
+        ) as mocked_obj:
+            await run_rulesets(
+                event_log,
+                ruleset_queues,
+                dict(),
+                "playbooks/inventory.yml",
+            )
+
+            while not event_log.empty():
+                event = event_log.get_nowait()
+                if event["type"] == "Action":
+                    action = event
+
+            assert action["url"] == job_url
+            assert action["action"] == action_type
+            assert (
+                list(mocked_obj.call_args.args[2]["extra_vars"].keys())
+                == expected_keys
+            )