docs: add 'inserting hosts' section in docs

[Alex-Izquierdo]

No description provided.

[mkanoor]

@Alex-Izquierdo we need to clarify another piece here

1. The insert_hosts_to_meta filter will silently suppress any issues if the key is missing which could be bad. For example in The run_job_template action is not passing the host limit correctly #510 the crux of the issue was host_path had an incorrect key, note that event is not needed.

     filters:
        - ansible.eda.insert_hosts_to_meta:
            host_path: event.payload.event.fields.source

The host_path should have been just payload.event.fields.source but the filter ignores this error
Which caused the customer to think it's broken and they tried different ways to get the hosts into the job_args and started using the limit.

So we need to document that event prefix is needed for conditions and action args but not for source args and source filters.
We could have handled it differently by adding an extra argument to the filter say
raise_exception: True
and have it terminate the rulebook if it was bad, once the customer was content that the hosts is there in the payload they could have turned it off with
raise_exception: False.
If the customer is putting the host_path in the filter they need to be assured that it is there in every payload or if they feel some payloads will have it and others won't they can use the raise_exception accordingly.

2. It would also have helped if we had put a logger.warn in the source filter so it was apparent from the beginning that there is something wrong.

[mkanoor]

Can we add some more notes on how the filter is suppressing key errors

[Alex-Izquierdo]

@Alex-Izquierdo we need to clarify another piece here

1. The insert_hosts_to_meta filter will silently suppress any issues if the key is missing which could be bad. For example in The run_job_template action is not passing the host limit correctly #510 the crux of the issue was host_path had an incorrect key, note that event is not needed.

     filters:
        - ansible.eda.insert_hosts_to_meta:
            host_path: event.payload.event.fields.source

The host_path should have been just payload.event.fields.source but the filter ignores this error Which caused the customer to think it's broken and they tried different ways to get the hosts into the job_args and started using the limit.

So we need to document that event prefix is needed for conditions and action args but not for source args and source filters. We could have handled it differently by adding an extra argument to the filter say raise_exception: True and have it terminate the rulebook if it was bad, once the customer was content that the hosts is there in the payload they could have turned it off with raise_exception: False. If the customer is putting the host_path in the filter they need to be assured that it is there in every payload or if they feel some payloads will have it and others won't they can use the raise_exception accordingly.

2. It would also have helped if we had put a logger.warn in the source filter so it was apparent from the beginning that there is something wrong.

I think we have to clarify the scope of this PR and the documentation of ansible-rulebook. What you say is not general, but specific to the insert_hosts_to_meta. This section is just to indicate the use case of the plugin regarding host limits, not to document the plugin itself.

Unless we decide that ansible-rulebook docs portal is going to store the documentation for the plugins of the EDA collection, such a documentation should be in the documentation for the plugins (wherever it is going to be) and not here otherwise it will create more confusion. Documentation that by the way, right now only exists in the comments of the plugin, we already have a task to create that documentation.

So I can remark here that it should not include the "event" prefix until we write the documentation of the plugins and/or update the plugin to handle better the issue that you have described, but we should not put here the whole documentation for that plugin.

We definitely should prioritize the documentation of the plugins.

In fact, we already have a "sources" section when we list the plugins (are outdated) in the collection which is confusing, untied the versioning of the collection and ansible-rulebook. I feel we need to revisit all of this.

[bzwei]

I created ansible/event-driven-ansible#197 to add the option to raise an exception from the insert_hosts_to_meta filter.

[Alex-Izquierdo]

I created ansible/event-driven-ansible#197 to add the option to raise an exception from the insert_hosts_to_meta filter.

@bzwei great! If we merge it first, I can update this.